-- Logs begin at Sat 2022-09-24 04:21:40 UTC, end at Sat 2022-09-24 05:22:51 UTC. -- Sep 24 04:46:13.620711 dv-cinder-859186-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 24 04:46:13.628283 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 24 04:46:13 2022] *** Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: nodename: dv-cinder-859186-1-iscsi Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: machine: x86_64 Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: clock source: unix Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: pcre jit disabled Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: detected number of CPU cores: 4 Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: current working directory: / Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: detected binary path: /usr/bin/uwsgi-core Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: your processes number limit is 47680 Sep 24 04:46:13.630956 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: your memory page size is 4096 bytes Sep 24 04:46:13.631609 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: detected max file descriptor number: 2048 Sep 24 04:46:13.631609 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: lock engine: pthread robust mutexes Sep 24 04:46:13.631609 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: thunder lock: enabled Sep 24 04:46:13.631609 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 24 04:46:13.631609 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 24 04:46:13.673321 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: Python main interpreter initialized at 0x558cb60de9c0 Sep 24 04:46:13.673321 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: python threads support enabled Sep 24 04:46:13.673321 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: your server socket listen backlog is limited to 100 connections Sep 24 04:46:13.673321 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: your mercy for graceful operations on workers is 90 seconds Sep 24 04:46:13.673664 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: mapped 403077 bytes (393 KB) for 2 cores Sep 24 04:46:13.673764 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: *** Operational MODE: preforking *** Sep 24 04:46:13.674087 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: *** uWSGI is running in multiple interpreter mode *** Sep 24 04:46:13.674087 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: spawned uWSGI master process (pid: 81381) Sep 24 04:46:13.674289 dv-cinder-859186-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 24 04:46:13.674867 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: spawned uWSGI worker 1 (pid: 81393, cores: 1) Sep 24 04:46:13.675208 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: spawned uWSGI worker 2 (pid: 81394, cores: 1) Sep 24 04:46:13.675365 dv-cinder-859186-1-iscsi devstack@keystone.service[81381]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 24 04:46:17.959935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 04:46:17.959935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: from cryptography.utils import int_from_bytes Sep 24 04:46:17.960681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 04:46:17.960681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: from cryptography.utils import int_from_bytes Sep 24 04:46:17.962184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 04:46:17.962184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: from cryptography.utils import int_from_bytes Sep 24 04:46:17.962640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 24 04:46:17.962640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: from cryptography.utils import int_from_bytes Sep 24 04:46:18.009844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 04:46:18.010051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 04:46:18.010806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] command line args: [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 04:46:18.011079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 04:46:18.011256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ================================================================================ {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 04:46:18.011507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] admin_token = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.011748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] config_dir = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.011933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.012114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] config_source = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.012293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] debug = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.012491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.012755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 24 04:46:18.012819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 24 04:46:18.012883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] command line args: [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 24 04:46:18.013090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 24 04:46:18.013323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ================================================================================ {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 24 04:46:18.013510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] admin_token = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_config_append = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.013963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] config_dir = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_dir = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] config_source = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_file = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] debug = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.014902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.015942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_config_append = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_dir = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_file = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.016931 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.017862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] publish_errors = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.018897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.019930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] standard_threads = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] use_journal = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] publish_errors = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] use_json = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.020984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] use_stderr = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] use_syslog = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.021793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.021906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.022104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.022221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] standard_threads = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.022332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.022442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.022553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.022669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.022790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.022908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.022908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] use_journal = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.023125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.023241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] use_json = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.023359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.023479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] use_stderr = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.023479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] use_syslog = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.023694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.023807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 24 04:46:18.023917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024146 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.024939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.025848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.026642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.026759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.026879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.026879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.027999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.028984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.029988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.030930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.031961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.032885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.033861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.034900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.035970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036155 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.036904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.037936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.038991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.039982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.040913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.041920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.042976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.043821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.044929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.045893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.046909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.external = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.047966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.password = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] auth.token = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.048936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.external = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.049964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.password = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] auth.token = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.050906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.051917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.052953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.053984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.054940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.055983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.056910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.057941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.058930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.059973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.060891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.061887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.062946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.063968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.064902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.065929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.066910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.067944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.068986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.069901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.070937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.071970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.072082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.072194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.072314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.072428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.072540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.072652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.073958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.074924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075385 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.075960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.076992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.077913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.078958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.079991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] role.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] role.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.080899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.081880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] role.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.082958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] role.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.083977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.084892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.085926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.086955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.087993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.088106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.088217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.088217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.089926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.090882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.091998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.connection = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.092805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.093933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.094997 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.095967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.096090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.096207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.096320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.096481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.096577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.connection = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.096679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 04:46:18.096776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.097159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.097632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.098117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.098582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.099058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.099531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.099901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.100167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.100419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.100678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.100928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.101248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.101707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.102108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.102357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.102612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.102860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 24 04:46:18.103039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 24 04:46:18.126266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.126495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.126723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.128396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.128631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.128860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.135837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.137824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.163657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.163888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.165788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.166047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.217234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.217372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.265816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.266080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.266168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.266318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.420924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.421884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.422049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.422175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.422335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.422551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.445896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.446693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.457673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.458059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.471115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.471299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.471528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.471748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.471913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.471996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.472836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.473053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 24 04:46:18.489162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.489378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 04:46:18.489562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.489727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.489809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 04:46:18.489934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 04:46:18.490126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.490248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.490332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 24 04:46:18.490514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.490594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.490777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.490866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.491039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.491119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.491324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.501870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.502101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.508947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.509116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.512438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.512599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.512728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.512884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.513986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.514168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.514282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.514446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.514553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.514724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.514824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.515002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.515084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.515277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.515357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.538279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.538904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.539770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.540059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.540388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.540671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.545348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.545639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.545925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.546030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.546245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.546524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.552385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.552679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.552927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.553212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.558451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.558758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.558909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.559050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.559208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.559346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.559500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.559631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.559784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.559916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.560064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.560193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.560338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.560466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.560608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.560875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.576757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.576953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.577049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.577232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.580973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.581058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.584277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.584374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.584573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.584655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.588854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.588939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.592058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.592214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.595326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.595518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.595628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.595823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.595906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.596985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.613213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.613500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.613582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.613764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.613845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.614045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.614129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.614404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.621055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.621436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.622889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.623308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.624420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.624695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.624853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.625137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.627499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.627784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.627929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.628068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.628218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.628348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.628510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.628733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.628815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.629183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.640780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.641070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.641163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.641352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.641434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.641639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.641729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.641930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.642037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.642311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.642392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.642591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.642685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.642893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.658711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.659116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.661615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.662090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.664656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.665027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.666184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.666582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.667701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.667984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.668066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.668322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.668404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.668728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.675079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.675208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.676538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.676693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.679455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.679673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.679850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.680076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.680157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.680370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.680465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.680654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.688300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.688591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.688673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.688870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.688955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.689217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.695915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.696199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.696293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 24 04:46:18.696476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.696559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.696750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.696833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.697019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.697111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.697289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.697372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.697560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.697640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.698397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.698489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.698690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.698772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.698955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.699918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.700048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.700205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.700329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 24 04:46:18.734862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558cb60de9c0 pid: 81393 (default app) Sep 24 04:46:18.735288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558cb60de9c0 pid: 81394 (default app) Sep 24 04:46:18.738093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04f155-9028-4f82-8d46-747a2a2cae32 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:18.738379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04f155-9028-4f82-8d46-747a2a2cae32 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:18.738648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04f155-9028-4f82-8d46-747a2a2cae32 None None] PATH_INFO: `/v3` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:18.743157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Sat Sep 24 04:46:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:25.506340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:25.506863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:25.506863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:25.511125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:46:25.516280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a69171-ae16-4480-9819-6c76d515086d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:25.516857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a69171-ae16-4480-9819-6c76d515086d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:25.516857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a69171-ae16-4480-9819-6c76d515086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:25.587102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: INFO dbcounter [None req-38a69171-ae16-4480-9819-6c76d515086d None None] Registered counter for database keystone Sep 24 04:46:25.606891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG oslo_db.sqlalchemy.engines [None req-38a69171-ae16-4480-9819-6c76d515086d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81393) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 04:46:25.613048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writer thread running {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 04:46:25.791281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG passlib.handlers.bcrypt [None req-38a69171-ae16-4480-9819-6c76d515086d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81393) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 04:46:25.792145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG passlib.handlers.bcrypt [None req-38a69171-ae16-4480-9819-6c76d515086d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81393) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 04:46:25.849026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-38a69171-ae16-4480-9819-6c76d515086d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:25.850657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-38a69171-ae16-4480-9819-6c76d515086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:25.976779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 04:46:26.449323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:26.518817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: INFO dbcounter [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] Registered counter for database keystone Sep 24 04:46:26.539780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG oslo_db.sqlalchemy.engines [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81394) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 24 04:46:26.545842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writer thread running {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 24 04:46:26.673056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53673d-5063-433c-943a-fc2bc45ca90f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:26.677321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:26.677948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:26.678256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:26.678530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:26.687230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:26.739344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING keystone.common.rbac_enforcer.enforcer [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 04:46:26.739690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a13299-bf13-46e3-a027-33e935879735 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:26.749142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Sat Sep 24 04:46:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 04:46:26.755071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a69171-ae16-4480-9819-6c76d515086d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:26.771958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a69171-ae16-4480-9819-6c76d515086d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:26.776962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:26.778091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:26.778370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:26.778646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] PATH_INFO: `/v3/projects/f9c9e5a50c664c33aa419611e20c9c00` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:26.780214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] RBAC: Authorizing `identity:get_project(project_id=f9c9e5a50c664c33aa419611e20c9c00)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:26.832965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 24 04:46:26.833305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec421849-58cd-4a8a-904b-acd2f943e1ca admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:26.837769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Sat Sep 24 04:46:26 2022] GET /identity/v3/projects/f9c9e5a50c664c33aa419611e20c9c00 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:28.952532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be103-8d13-49b2-baa3-958ed920e78c None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:28.953061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be103-8d13-49b2-baa3-958ed920e78c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:28.953061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8be103-8d13-49b2-baa3-958ed920e78c None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:28.957323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:46:28.963862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:28.964369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:28.964461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:28.987667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:28.989193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:28.997040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 24 04:46:29.269460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be103-8d13-49b2-baa3-958ed920e78c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:29.281406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8be103-8d13-49b2-baa3-958ed920e78c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:29.285527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711771e-1f29-4722-b5e7-132dd9006dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:29.286152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711771e-1f29-4722-b5e7-132dd9006dcf admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:29.286430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711771e-1f29-4722-b5e7-132dd9006dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:29.286701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711771e-1f29-4722-b5e7-132dd9006dcf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:29.294608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711771e-1f29-4722-b5e7-132dd9006dcf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:29.296392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711771e-1f29-4722-b5e7-132dd9006dcf admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:29.304939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Sat Sep 24 04:46:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 04:46:29.310754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:29.319787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c46cbb-d343-4bba-a8cd-261db68635ed None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:29.324201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5b1c5-cdb8-4c0f-a58e-9d16fe944b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:29.324820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5b1c5-cdb8-4c0f-a58e-9d16fe944b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:29.325100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5b1c5-cdb8-4c0f-a58e-9d16fe944b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:29.325374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5b1c5-cdb8-4c0f-a58e-9d16fe944b60 admin admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:29.326707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5b1c5-cdb8-4c0f-a58e-9d16fe944b60 admin admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:29.328517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5b1c5-cdb8-4c0f-a58e-9d16fe944b60 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:29.330212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Sat Sep 24 04:46:29 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:31.520700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb147-0a1c-4748-b309-a36599e2737f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:31.521238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb147-0a1c-4748-b309-a36599e2737f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:31.521329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdb147-0a1c-4748-b309-a36599e2737f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:31.522768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:31.528132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:31.528553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:31.528799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:31.547112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:31.548387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:31.549682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1898c6-fd0f-479d-b782-5e3005b19d64 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:31.549963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1898c6-fd0f-479d-b782-5e3005b19d64 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:31.550280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1898c6-fd0f-479d-b782-5e3005b19d64 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:31.551574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:46:31.556736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:31.559468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:31.559753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:31.601016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG passlib.handlers.bcrypt [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81394) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 24 04:46:31.601396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG passlib.handlers.bcrypt [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81394) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 24 04:46:31.630016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:31.651216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:31.652807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:31.751482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:31.899658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:31.923995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.003625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae1a2d-ccfc-4f81-869b-ada26a9206fb None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.014982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.037593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.038264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.038548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.038827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.050908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:32.062716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:32.063825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:46:32.064743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:46:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:32.069953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.115924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48a553-c9d6-4ea2-aae9-89a63f188e48 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.118245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.151344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.154989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.155854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.156133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.156406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.169577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e4498-c4d4-4e60-b56e-d09d9a8e4fb6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.202171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.202773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.203043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.203313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.203974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:32.204268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.205639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.217484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:32.221904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.223054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.223707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.317418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 8/16] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:46:32.317585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2dcdb-7227-4240-aa90-c91e77fa9f7c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.323449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.326112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.354513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.355147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.355427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.355700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.356843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.358355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.359920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:32.365243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.380005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.423771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d00301-ba3a-459f-b8b2-9f55675eb831 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.458889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.459734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.460008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.460277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.473026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60938295-d4ce-4f15-8363-f5a967d41b12 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.478542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.479996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:46:32.499439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:32.500514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:46:32.501640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:46:32.501640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:46:32.501640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:32.501640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 9/18] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:46:32.506300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.510224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.510857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.511136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.511409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.512524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.513967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.515517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 10/19] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:32.520999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:32.563095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.600326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:32.607512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcdbb5-ba1b-481a-ac01-b777e4783e6d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.620182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44ea4-ab2b-448f-b5e3-7883d0e37ae8 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:32.640031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.640633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.640909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.641179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.641814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:32.642143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.643534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0afd9fd-51ef-40cd-aa5e-bb40c92a5ad5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.656529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:32.657136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:32.657414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:32.657687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:32.658456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.domain.id=default, user.id=eb0ba9d6a6bf45098fbc94fef46dc27d) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:32.658761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:32.660175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b23557-d6bd-4b2d-b24e-f7076d8c168b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:32.661588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 10/20] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:32.667805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 11/21] 10.222.0.50 () {56 vars in 1322 bytes} [Sat Sep 24 04:46:32 2022] GET /identity/v3/role_assignments?user.id=eb0ba9d6a6bf45098fbc94fef46dc27d&scope.domain.id=default&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:35.096381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d77d-84f8-4dee-a0c5-49176bc75f2e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.096915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d77d-84f8-4dee-a0c5-49176bc75f2e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.096915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236d77d-84f8-4dee-a0c5-49176bc75f2e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.098304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 11/22] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:35.103376 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29b35-238a-48b0-9bc4-5f4d6cde20ee None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.104285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29b35-238a-48b0-9bc4-5f4d6cde20ee None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.104534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f29b35-238a-48b0-9bc4-5f4d6cde20ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.131853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b2f29b35-238a-48b0-9bc4-5f4d6cde20ee None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:35.133059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b2f29b35-238a-48b0-9bc4-5f4d6cde20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:35.219455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 12/23] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:35.412565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5cbb4-035b-4b3e-8b3d-08736b5d5f44 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.413048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5cbb4-035b-4b3e-8b3d-08736b5d5f44 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.413486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5cbb4-035b-4b3e-8b3d-08736b5d5f44 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.415318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 12/24] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:35.420403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c242f1-0601-4a4f-99aa-90d483d9240a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.420694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c242f1-0601-4a4f-99aa-90d483d9240a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.420980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c242f1-0601-4a4f-99aa-90d483d9240a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.448931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-17c242f1-0601-4a4f-99aa-90d483d9240a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:35.450219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-17c242f1-0601-4a4f-99aa-90d483d9240a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:35.554677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 13/25] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:35.607606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f290d0-1f14-4d85-abe7-442bf68fc290 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.608206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f290d0-1f14-4d85-abe7-442bf68fc290 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.608698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f290d0-1f14-4d85-abe7-442bf68fc290 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.610299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 13/26] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:35.615379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.615664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.615939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.625282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:35.642137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:35.643368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:35.725103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f290d0-1f14-4d85-abe7-442bf68fc290 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:35.731244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 14/27] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:35.887722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f290d0-1f14-4d85-abe7-442bf68fc290 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:35.888802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:35.907481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:35.944282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:35.944941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:35.945223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:35.945497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:35.975217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:35.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:35.988036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:35.988781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 14/28] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:46:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:46:35.995977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.109551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a487-0f83-441c-aca0-eafeca6fc2cd None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.176140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.177086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.177630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.177910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.180308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:36.183050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60176fae-ae76-408c-840c-0d2691e4ea47 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.191022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.193062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-b7f9333d-33e5-4bf5-96af-cd12a30e0fc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:46:36.214306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:36.215340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:46:36.216292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:46:36.216292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:46:36.216292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:46:36.218224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 15/29] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:36.219581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ca730-f10b-4f41-a008-b7212b4a5570 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.220026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ca730-f10b-4f41-a008-b7212b4a5570 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.220460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ca730-f10b-4f41-a008-b7212b4a5570 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.222216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 16/30] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:36.223186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ca730-f10b-4f41-a008-b7212b4a5570 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.231719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.232608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.233034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.233451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.234308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:36.234745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.236397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.254193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 15/31] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:36.255262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.303582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:36.343821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ca730-f10b-4f41-a008-b7212b4a5570 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.378638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a31310-745a-438d-8eb5-82564a975284 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.379648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.380410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.380820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.381243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.382193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.383809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.415616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.416306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.417119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.417394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.418076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:36.418369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.419739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38133752-cabc-4c50-bf32-9c30761ae973 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.431355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 16/32] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:46:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:36.432601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.432885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.433156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.456971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:36.459025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:36.529961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 17/33] 10.222.0.50 () {60 vars in 1131 bytes} [Sat Sep 24 04:46:36 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:36.531548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.538055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 17/34] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:36.538891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.586979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:36.588907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:36.655716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664433c7-9dfd-4084-b6e3-9be979a4c93e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.698524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.699611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.700077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.700514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.702049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.703985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.705889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 18/35] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:36 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:36.711483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.739269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2e7d2-b807-4c6c-af0b-baf5fbe2683c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.784346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596bc2a-966e-4e51-be25-2f3b55d6d92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.785261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596bc2a-966e-4e51-be25-2f3b55d6d92b None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.786245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596bc2a-966e-4e51-be25-2f3b55d6d92b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.786716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596bc2a-966e-4e51-be25-2f3b55d6d92b None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.788453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596bc2a-966e-4e51-be25-2f3b55d6d92b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.790477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596bc2a-966e-4e51-be25-2f3b55d6d92b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.881981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8929820-bcf8-46d6-b2cf-59e543458690 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:36.898027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:36.922325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7920ce3-dda6-4c7b-a49e-3ae400796a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:36.923018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7920ce3-dda6-4c7b-a49e-3ae400796a11 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.923345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7920ce3-dda6-4c7b-a49e-3ae400796a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.923617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7920ce3-dda6-4c7b-a49e-3ae400796a11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.924904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7920ce3-dda6-4c7b-a49e-3ae400796a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:36.926408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7920ce3-dda6-4c7b-a49e-3ae400796a11 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:36.930613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 19/36] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:36.935293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c42d57-e264-41c7-ada1-786a666a1e72 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.935587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c42d57-e264-41c7-ada1-786a666a1e72 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.935924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c42d57-e264-41c7-ada1-786a666a1e72 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:36.937492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 20/37] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:36.942907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c42d57-e264-41c7-ada1-786a666a1e72 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:36.980782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 18/38] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:46:36 2022] POST /identity/v3/projects => generated 301 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:36.982827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:36.983403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:36.983981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.015438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:37.018472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:37.080412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c42d57-e264-41c7-ada1-786a666a1e72 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:37.110186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 19/39] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:37.111478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:37.120439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf486285-c4a2-4fc5-8da3-18716da3384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:37.121127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf486285-c4a2-4fc5-8da3-18716da3384d None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.121398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf486285-c4a2-4fc5-8da3-18716da3384d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.121679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf486285-c4a2-4fc5-8da3-18716da3384d None admin] PATH_INFO: `/v3/domains/default/users/eb0ba9d6a6bf45098fbc94fef46dc27d/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.124636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf486285-c4a2-4fc5-8da3-18716da3384d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=eb0ba9d6a6bf45098fbc94fef46dc27d, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:37.126558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf486285-c4a2-4fc5-8da3-18716da3384d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:37.127770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:37.147402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 21/40] 10.222.0.50 () {58 vars in 1361 bytes} [Sat Sep 24 04:46:36 2022] PUT /identity/v3/domains/default/users/eb0ba9d6a6bf45098fbc94fef46dc27d/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 04:46:37.152031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcf871-80f8-440d-a39c-3766b61593f0 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.152659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcf871-80f8-440d-a39c-3766b61593f0 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.153177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcf871-80f8-440d-a39c-3766b61593f0 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.156151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 22/41] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:37.163640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.164254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.165762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.277625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fc21b-af1a-4643-8f57-1b07850e7afc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:37.312892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:37.314594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:37.331293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:37.331293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.331564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.331630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.332324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:37.333895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:37.357775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 20/42] 10.222.0.50 () {60 vars in 1131 bytes} [Sat Sep 24 04:46:37 2022] POST /identity/v3/roles => generated 219 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:37.370980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:37.403619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:37.421070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 23/43] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:37.492542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:37.535822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8524e-b783-420d-8f7f-dcc3b50fb3b9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:37.575970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:37.576598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.576883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.577161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.577871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:37.579328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:37.595950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 21/44] 10.222.0.50 () {60 vars in 1131 bytes} [Sat Sep 24 04:46:37 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:46:37.702255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:37.809129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc04fa0-40f8-4218-a8da-81b7434fdb32 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:37.854085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc5d92-6a24-491b-a404-2c5b69f8c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:37.854659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc5d92-6a24-491b-a404-2c5b69f8c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.854938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc5d92-6a24-491b-a404-2c5b69f8c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.855210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc5d92-6a24-491b-a404-2c5b69f8c2a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.856305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc5d92-6a24-491b-a404-2c5b69f8c2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:37.857750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc5d92-6a24-491b-a404-2c5b69f8c2a6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:37.859366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 24/45] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:37.864768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:37.962400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045690c-b938-4a65-9ce8-9849f0f563ed None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:37.990232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:37.997424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5700c08-2c85-4d18-8938-9db3f6e5215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:37.998324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700c08-2c85-4d18-8938-9db3f6e5215c None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:37.998608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700c08-2c85-4d18-8938-9db3f6e5215c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:37.998886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5700c08-2c85-4d18-8938-9db3f6e5215c None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:37.999550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5700c08-2c85-4d18-8938-9db3f6e5215c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:38.001012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5700c08-2c85-4d18-8938-9db3f6e5215c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:38.026866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 22/46] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:46:37 2022] POST /identity/v3/projects => generated 312 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:39.482649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498436b-e750-458b-8d5f-5ea8f919da19 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:39.483163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498436b-e750-458b-8d5f-5ea8f919da19 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:39.483244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498436b-e750-458b-8d5f-5ea8f919da19 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:39.484539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 25/47] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:39.489776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:39.490206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:39.490376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:39.508696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:39.509871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:39.574235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 23/48] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:39.844129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498436b-e750-458b-8d5f-5ea8f919da19 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:39.886573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:39.945269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498436b-e750-458b-8d5f-5ea8f919da19 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:39.978118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515d451-f629-4f72-9619-090a838d8506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:39.979310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515d451-f629-4f72-9619-090a838d8506 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:39.979596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515d451-f629-4f72-9619-090a838d8506 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:39.979865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515d451-f629-4f72-9619-090a838d8506 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:39.989634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515d451-f629-4f72-9619-090a838d8506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-e515d451-f629-4f72-9619-090a838d8506 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:39.994538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:39.995485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:46:39.996404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 26/49] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:46:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:40.003242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:40.190554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04fcb-f6c3-4d7a-83f5-412e4a29c104 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.190738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04fcb-f6c3-4d7a-83f5-412e4a29c104 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.191118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04fcb-f6c3-4d7a-83f5-412e4a29c104 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.192462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 27/50] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:46:40.197492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.198460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.198735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.204897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033a7e2-6c99-49c2-8ec8-50486e2af430 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:40.216193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:40.217398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:40.217601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:40.241476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:40.242120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.242403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.242676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.243340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:40.243645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:40.245609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:40.254088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 24/51] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:40.259532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:40.284057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 28/52] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:40.358658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cffa08-8172-43a1-9eb2-6367ccfbe099 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:40.391278 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:40.391910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.392190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.392463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.393818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:40.395299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:40.396875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 25/53] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:40 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:40.403013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:40.432325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:40.502786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365a718-eed6-4e8b-bb5f-b1d995db5255 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:40.535673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:40.536299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.536579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.536850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.537941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:40.539407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:40.540954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 29/54] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:40.547328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:40.551477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:40.591067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:40.648327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:40.648484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e3e8a-a57d-4169-8224-b730eee77c38 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:40.650438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f73453-abfd-437f-aebd-a9fb25ef78da None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:40.682385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:40.683001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.683281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.683553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.684283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.domain.id=default, user.id=eb0ba9d6a6bf45098fbc94fef46dc27d) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:40.684588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:40.686032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:40.686157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3689d09-8156-4b7b-8e1e-d7e3ed1de80e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:40.686668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.686948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.687224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.688324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:40.689740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:40.691337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 30/55] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:40.694841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 26/56] 10.222.0.50 () {56 vars in 1322 bytes} [Sat Sep 24 04:46:40 2022] GET /identity/v3/role_assignments?user.id=eb0ba9d6a6bf45098fbc94fef46dc27d&scope.domain.id=default&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:40.696729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:40.807687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844ccac-4d22-461c-a7c1-3b75b0568f7b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:40.843287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19c4f7-b137-4c77-ada7-61bea9e34858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:40.843913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c4f7-b137-4c77-ada7-61bea9e34858 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:40.844193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c4f7-b137-4c77-ada7-61bea9e34858 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:40.844467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19c4f7-b137-4c77-ada7-61bea9e34858 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:40.845143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19c4f7-b137-4c77-ada7-61bea9e34858 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:40.846650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19c4f7-b137-4c77-ada7-61bea9e34858 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:40.876096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 31/57] 10.222.0.50 () {60 vars in 1140 bytes} [Sat Sep 24 04:46:40 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:43.071268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2f54d-293a-4206-8d79-1eb9a083753a None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:43.071801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2f54d-293a-4206-8d79-1eb9a083753a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:43.071801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2f54d-293a-4206-8d79-1eb9a083753a None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:43.073729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 27/58] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:43.078835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:43.079214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:43.079387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:43.097489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:43.098705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:43.111380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:43.164459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 32/59] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:43.430479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2f54d-293a-4206-8d79-1eb9a083753a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:43.532963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2f54d-293a-4206-8d79-1eb9a083753a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:43.540090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:43.566773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6bdf1-0815-49f8-a6f2-ecf339758dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:43.567395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bdf1-0815-49f8-a6f2-ecf339758dbc None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:43.567671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bdf1-0815-49f8-a6f2-ecf339758dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:43.567941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bdf1-0815-49f8-a6f2-ecf339758dbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:43.569038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6bdf1-0815-49f8-a6f2-ecf339758dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:43.570528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6bdf1-0815-49f8-a6f2-ecf339758dbc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:43.572080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 28/60] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:43.577610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:43.673912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67b193-9dc0-4ae1-9237-35118770e453 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:43.706069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e95341-a1c1-4ac7-ae42-b41f2d0a7d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:43.706639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e95341-a1c1-4ac7-ae42-b41f2d0a7d94 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:43.706921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e95341-a1c1-4ac7-ae42-b41f2d0a7d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:43.707197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e95341-a1c1-4ac7-ae42-b41f2d0a7d94 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:43.707838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e95341-a1c1-4ac7-ae42-b41f2d0a7d94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:43.709276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e95341-a1c1-4ac7-ae42-b41f2d0a7d94 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:43.735839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 33/61] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:46:43 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:47.126832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97799c-b4fe-4a80-a9f6-97ef5789e296 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.127349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97799c-b4fe-4a80-a9f6-97ef5789e296 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.127349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97799c-b4fe-4a80-a9f6-97ef5789e296 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.128952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 29/62] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:47.134468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcaa44-cb97-4d57-9dab-ecfaf03fbe28 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.135402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcaa44-cb97-4d57-9dab-ecfaf03fbe28 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.135606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcaa44-cb97-4d57-9dab-ecfaf03fbe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.148070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:47.152598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488695-6217-4cad-822f-5558b63576f2 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.153082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488695-6217-4cad-822f-5558b63576f2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.153492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0488695-6217-4cad-822f-5558b63576f2 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.154972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 30/63] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:46:47.162158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dced4-3c40-4043-8d71-16d88445b514 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.162699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dced4-3c40-4043-8d71-16d88445b514 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.163160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3dced4-3c40-4043-8d71-16d88445b514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.173965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-cdfcaa44-cb97-4d57-9dab-ecfaf03fbe28 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:47.176313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cdfcaa44-cb97-4d57-9dab-ecfaf03fbe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:47.197966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-fb3dced4-3c40-4043-8d71-16d88445b514 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:47.199524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fb3dced4-3c40-4043-8d71-16d88445b514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:47.269686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 34/64] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:47.271421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9dd44-b94f-4dfe-aa96-8a7ff82147d7 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.271872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9dd44-b94f-4dfe-aa96-8a7ff82147d7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.272328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9dd44-b94f-4dfe-aa96-8a7ff82147d7 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.273745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 35/65] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:47.279468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476aa5c-4085-4d32-b0af-215f91e478b7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.279974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476aa5c-4085-4d32-b0af-215f91e478b7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.280424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476aa5c-4085-4d32-b0af-215f91e478b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.300740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 31/66] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:47.304750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b476aa5c-4085-4d32-b0af-215f91e478b7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:47.305950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b476aa5c-4085-4d32-b0af-215f91e478b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:47.375503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 36/67] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:47.568363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2208-51cd-43c5-b1f3-0c98300157aa None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.568881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2208-51cd-43c5-b1f3-0c98300157aa None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.569298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2208-51cd-43c5-b1f3-0c98300157aa None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.570830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 32/68] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:47.576115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.576555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.576964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.595854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:47.597252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:47.624489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722558c-51ee-4779-9436-8752c0da27ac None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.624981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722558c-51ee-4779-9436-8752c0da27ac None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.625391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722558c-51ee-4779-9436-8752c0da27ac None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.626874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 33/69] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:46:47.631880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.632311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.632761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.660247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:47.661438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:47.703050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:47.713898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 37/70] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:47.714996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:47.734037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 34/71] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:47.735123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:47.785598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:47.821890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c7cf8-7752-400d-b171-6d2548f56f50 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:47.862809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:47.863660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.863903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a5510-9c39-4cd9-bc18-d1a8a95ca2b6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:47.864236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.864651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.877942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:47.884341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:47.885918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:46:47.886728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:46:47.888054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 38/72] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:46:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:47.890801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:47.907174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:47.907820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:47.908100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:47.908374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:47.921444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:47.927745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:47.928628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:47.929392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 35/73] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:47.930127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:47.966559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.023287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f37c2a-2d2d-497f-8657-f2be3b3629ca None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.040336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.047904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be92fd-3016-4679-9fb5-392581c6a387 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.056882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.057561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.057846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.058159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.067933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:48.073403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:48.074243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:46:48.075015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 39/74] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:48.075634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.083491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.084101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.084384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.084659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.085321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:48.085623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.087048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.096281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 36/75] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:46:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.096927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.177608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ef3cd-1a5b-4378-861b-5d09b63efa61 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.191658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.197307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ef137-7506-4b43-8a3a-ebc3b7790c2a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.211723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.212328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.212610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.212885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.214086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:48.214392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.215781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.223802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 40/76] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.224579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.231889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.232526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.232809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.233489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.243868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:48.250073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:48.250918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:48.251703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 37/77] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:48.252353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.270463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.325387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df8889-baa3-4efa-9440-f674683f47c5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.350347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2d4ae-8b83-4231-adcf-c6a8cf221a37 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.358594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.359260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.359545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.359820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.360935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.362423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.363990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 41/78] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.364748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.384982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.385595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.385880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.386205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.386872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:48.387179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.388558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.396599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 38/79] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.397303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.415352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.470189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234673c-46aa-4481-840e-0d0874b7cb5d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.474965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.495953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986fed4-4a77-4ab7-ac55-096af737a8f5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.502501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.503126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.503407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.503682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.517344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.518839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.520440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 42/80] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.521201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.534220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.534862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.535149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.535428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.536574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.538041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.539616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 39/81] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.544104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.621465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566e65b-1b89-4989-91a5-b2c8f642e8f8 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.623353 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.643372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e109db0-b2f1-433b-a980-0fb766d1a872 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.657400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.658041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.658326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.658599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.659260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:48.659559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.660959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.668917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 43/82] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.669686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.678286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.678895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.679172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.679446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.680090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.681531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.708464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 40/83] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:46:48 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:48.709582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.712184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.789722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4184770-6209-41a5-97bc-e30194ae47b6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.827799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.828449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.828731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.829007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.830205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.831677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.832026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b0c67-8b2b-4932-bb91-2240a2179803 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.833559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 44/84] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:48.834194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:48.868927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:48.878296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:48.878852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:48.879138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:48.879412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:48.956727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d8eee-1489-4bc5-8d2f-93f83c746b00 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:48.962554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:48.964744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:48.966776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.000158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.000779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.001061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.001334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.076711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.078244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.094746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 41/85] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:46:49.095596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.126433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.199107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 45/86] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:46:49.200789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.219671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa944ab-17a2-4f35-902b-173e0b7c4608 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:49.221859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.260821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.261745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.262251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.262695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.264233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.265982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.268006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 42/87] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:49.269095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.308776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b65fa-e8eb-4545-9b23-86030bea933e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:49.339766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.345386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.346284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.346803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.347312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.385567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d048c-4e05-439b-8d48-4aebf6a408de None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:49.418537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.420318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.424388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.425066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.425353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.425630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.426787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.428241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.429881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 43/88] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:49.430740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.473524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.532126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 46/89] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:46:49.532986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.551672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae0f43-ec09-4776-82fd-94c104eedfef None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:49.570349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.594153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.595356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.595641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.595918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.597482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.599089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.601683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 44/90] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:49.602595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.652655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76942549-043d-42f8-8754-3dccd5a3619a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:49.691763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.692382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.692661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.692932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.728904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f2357-64d3-4017-8999-64216708e7b7 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:49.737925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.772925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:49.774026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:49.774515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:49.774968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:49.776258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] RBAC: Adding query filter params (role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:49.777300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.779522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.785183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:49.786712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:49.798475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 45/91] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/role_assignments?user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=399f7b70c70b4f09addcf156bf604eab => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:49.799762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:49.863061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:49.924996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603af69-f2d4-4bf8-91aa-060229e1c290 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:50.003654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:50.004282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:50.004564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:50.007172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:50.009798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:50.012044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:50.015129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 46/92] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:50.015129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:50.025064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 47/93] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:46:50.026377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:50.073144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:50.152984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcae3c6-1ce8-4c05-a1b1-c150cc4e94b2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:50.189087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:50.189709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:50.190012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:50.190318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:50.191054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:50.191469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:50.192952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:50.199259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 47/94] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:46:50 2022] GET /identity/v3/role_assignments?user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=6768099ea599447f9de57814f623441c => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:50.200084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:50.232202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348029a-c438-44e7-9f65-5c8bb032e72e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:50.292159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:50.314719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:50.315562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:50.315950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:50.316054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62165d5-0547-4003-b83b-23bbe7c4c228 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:50.316458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:50.316994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=eb0ba9d6a6bf45098fbc94fef46dc27d) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:50.317348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:50.317873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:50.319030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:50.330723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 48/95] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:46:50 2022] GET /identity/v3/role_assignments?user.id=eb0ba9d6a6bf45098fbc94fef46dc27d&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:50.364106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c7550-20d1-4eee-8af2-6970e42d63a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:50.364779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c7550-20d1-4eee-8af2-6970e42d63a7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:50.365064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c7550-20d1-4eee-8af2-6970e42d63a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:50.365346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c7550-20d1-4eee-8af2-6970e42d63a7 None admin] PATH_INFO: `/v3/projects/2d0db923f3214483b6ea510ef5d735ea` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:50.369134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c7550-20d1-4eee-8af2-6970e42d63a7 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0db923f3214483b6ea510ef5d735ea)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:50.370757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c7550-20d1-4eee-8af2-6970e42d63a7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:50.372971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 48/96] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:50 2022] GET /identity/v3/projects/2d0db923f3214483b6ea510ef5d735ea => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:50.379182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:50.516275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa254e5-1873-4d23-bfe1-20c13e9c346d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:50.563613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:50.564162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:50.564449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:50.564723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:50.565476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, scope.project.id=2d0db923f3214483b6ea510ef5d735ea, user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:50.565784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:50.567314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47877798-0b9f-4453-befe-4335139c4d1c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:50.575861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 49/97] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:46:50 2022] GET /identity/v3/role_assignments?user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da&scope.project.id=2d0db923f3214483b6ea510ef5d735ea&role.id=6768099ea599447f9de57814f623441c => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:51.349283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef29a7-dfac-451a-9648-8814d7a18cf9 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:51.349795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef29a7-dfac-451a-9648-8814d7a18cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:51.349897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef29a7-dfac-451a-9648-8814d7a18cf9 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:51.352268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 49/98] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:51.357521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744044b4-1279-4048-8adc-61d813bd7a85 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:51.357963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744044b4-1279-4048-8adc-61d813bd7a85 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:51.358171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744044b4-1279-4048-8adc-61d813bd7a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:51.387499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-744044b4-1279-4048-8adc-61d813bd7a85 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:51.389206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-744044b4-1279-4048-8adc-61d813bd7a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:51.523508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:51.544649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 50/99] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:51.586405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aded7-39c9-4377-b14b-dc4b0629e0dc None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:51.586710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aded7-39c9-4377-b14b-dc4b0629e0dc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:51.586964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60aded7-39c9-4377-b14b-dc4b0629e0dc None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:51.588493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 50/100] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:51.594009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:51.594532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:51.595007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:51.647090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:51.648634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:51.768411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 51/101] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:52.160739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60aded7-39c9-4377-b14b-dc4b0629e0dc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:52.234038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:52.324766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60aded7-39c9-4377-b14b-dc4b0629e0dc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:52.391680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:52.392425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:52.392608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:52.392880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:52.410153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:52.415700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:52.417236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:46:52.418193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:52.420192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 51/102] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:52.426085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:52.446236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f8c64-da05-4c00-8488-05dd4e081100 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:52.499626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:52.510692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:52.541680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:52.542638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:52.543118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:52.543620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:52.545751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:52.547764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:52.549562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 52/103] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:52.555358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:52.687775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa53b8-9b6e-4438-9d03-f5b46377d5fe None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:52.751229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da121dd-9118-4c27-a8c2-1c9270569873 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:52.776251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:52.777151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:52.777431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:52.777702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:52.778507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:52.778805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:52.780192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:52.797769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 52/104] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:46:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:52.804157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:52.810705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61fa97-b320-4b92-b69e-1532a584a209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:52.811714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61fa97-b320-4b92-b69e-1532a584a209 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:52.812219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61fa97-b320-4b92-b69e-1532a584a209 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:52.812693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61fa97-b320-4b92-b69e-1532a584a209 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:52.813590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61fa97-b320-4b92-b69e-1532a584a209 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:52.815353 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61fa97-b320-4b92-b69e-1532a584a209 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:52.864395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 53/105] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:46:52 2022] POST /identity/v3/projects => generated 302 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:52.970256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:52.982305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab162df-10cf-4f2c-8c08-ebdc92965aa7 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:53.026923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130da12-efa2-4017-9c31-039efd3d8761 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.027549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130da12-efa2-4017-9c31-039efd3d8761 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.028130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130da12-efa2-4017-9c31-039efd3d8761 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.034222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 54/106] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:53.037774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:53.038214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.038444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.038725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.038997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] PATH_INFO: `/v3/users/961970d1d0c9409dad8d940af31d9ef4` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.039302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.039857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.067786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] RBAC: Authorizing `identity:get_user(user_id=961970d1d0c9409dad8d940af31d9ef4)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:53.069222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:53.069332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:53.074327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 53/107] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:52 2022] GET /identity/v3/users/961970d1d0c9409dad8d940af31d9ef4 => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:53.075695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:53.082031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:53.136566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:53.199559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 55/108] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:53.255143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5429e6c0-3ec8-4b55-b2a4-d7c6da10ca99 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:53.301323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:53.301937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.302324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.302607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.325113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:53.384610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:53.386683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:53.505319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 54/109] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:46:53.512329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:53.630689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:53.693898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b63bc5-69ba-43a1-a985-0c03d59a2f7b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:53.745089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:53.785048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:53.785630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.785913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.786216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.788531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:53.789982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:53.793912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 56/110] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:53 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:53.799773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:53.846959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792800b7-6aec-483b-8a4b-be419739ff88 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:53.860239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:53.888203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbec875-9cac-4499-9e82-a893873acf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:53.888841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbec875-9cac-4499-9e82-a893873acf1e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.889128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbec875-9cac-4499-9e82-a893873acf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.889405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbec875-9cac-4499-9e82-a893873acf1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.900371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbec875-9cac-4499-9e82-a893873acf1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-0fbec875-9cac-4499-9e82-a893873acf1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:53.905905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:53.906798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:53.907535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 55/111] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:46:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:53.908948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2a994-c4a6-41ea-b022-eca8bf252709 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.908948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2a994-c4a6-41ea-b022-eca8bf252709 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.908948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2a994-c4a6-41ea-b022-eca8bf252709 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.914060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 56/112] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:53.914060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35b4be-097b-42a5-bac8-a3b1ab6aa0e8 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.914060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35b4be-097b-42a5-bac8-a3b1ab6aa0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.914060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35b4be-097b-42a5-bac8-a3b1ab6aa0e8 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.917092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 57/113] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:53.918085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcd7cd-8821-46eb-827d-b8e2ca2ea620 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.918376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcd7cd-8821-46eb-827d-b8e2ca2ea620 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.918656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcd7cd-8821-46eb-827d-b8e2ca2ea620 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.920430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 58/114] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:53.922216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b34d3-15d1-4597-b59e-24f60caaa23e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.922216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b34d3-15d1-4597-b59e-24f60caaa23e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.922216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b34d3-15d1-4597-b59e-24f60caaa23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.934697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f685985-e79b-4aac-a98d-23528b5b72f0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:53.939847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4a9b34d3-15d1-4597-b59e-24f60caaa23e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:53.941043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4a9b34d3-15d1-4597-b59e-24f60caaa23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:53.970680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:53.971291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:53.971577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:53.971855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:53.972582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] RBAC: Adding query filter params (role.id=e99d1aaf2c754f72a1b1f8c6bba548e1, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=961970d1d0c9409dad8d940af31d9ef4) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:53.972889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:53.974341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:53.990040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 57/115] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:46:53 2022] GET /identity/v3/role_assignments?user.id=961970d1d0c9409dad8d940af31d9ef4&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:53.990040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.031102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 59/116] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:54.032411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e366af-6379-415f-a9f9-3608fe5ab0db None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.032757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e366af-6379-415f-a9f9-3608fe5ab0db None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.033036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e366af-6379-415f-a9f9-3608fe5ab0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.056584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-67e366af-6379-415f-a9f9-3608fe5ab0db None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:54.059110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-67e366af-6379-415f-a9f9-3608fe5ab0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:54.139666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:54.158413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9a70f-09d7-4c8d-a5d5-4551dee768a1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:54.167399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:54.199892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 60/117] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:54.206026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.206831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.207310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.225183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:54.225853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.226178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.226510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.227196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:54.227550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:54.229730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:54.239459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:54.240947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:54.252081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 58/118] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:46:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:54.258303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.365310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 61/119] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:54.366827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.466948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e29ae-0612-41c9-9038-c81b736a279d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:54.550646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:54.551295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.551588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.551842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.553649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:54.555126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:54.559127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 59/120] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:54 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:46:54.574091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.610889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:54.613241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:54.614061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ff2a-952e-446e-8946-fb58b9c7a1bf None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:54.664493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:54.665042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.665319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.665593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.688623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:54.700604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:54.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:46:54.702295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:54.703648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 62/121] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:54.706523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.755171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b881c-4c18-4332-a546-1e760413d4fe None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:54.798531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:54.799224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.799559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.800345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.801960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:54.803885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:54.806323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 60/122] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:54 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:54.807554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.829255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f81a7-5ed7-4607-a5e0-1d223a1efe34 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:54.875179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:54.875816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.876366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.876643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.890366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:54.895456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:54.897521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 63/123] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:54.899415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:54.924533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:54.927674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b15ad-6704-4b55-9854-33c1766f5b2f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:54.933704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:54.965411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:54.966297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:54.966677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:54.967048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:54.978372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:54.983863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:54.984809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:46:54.985580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 61/124] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:54.991446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.033665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e581702-84ed-4741-a021-223d2d6eb528 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.080950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.082158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.082621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.083346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.084642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:55.085123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.087375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.099707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 64/125] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.099860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946fb46-8eca-4ac6-af6c-67254e70bd1b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.100908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.141689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.142471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.142821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.143167 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573/users/c6c71c8a128a4b5f93b4e8d9cf8f37da/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.145595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] RBAC: Authorizing `identity:create_grant(project_id=595b8e1093aa49399a51e159588da573, user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da, role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.147159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.163839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 62/126] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:46:54 2022] PUT /identity/v3/projects/595b8e1093aa49399a51e159588da573/users/c6c71c8a128a4b5f93b4e8d9cf8f37da/roles/399f7b70c70b4f09addcf156bf604eab => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:46:55.165090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.196257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:55.201124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:55.226951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8c2975-17dd-4eb6-94b9-19d12a0b20e4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.293677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.294663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.295131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.295604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.296417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] RBAC: Adding query filter params (name=member) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:55.297117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.300799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.318770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 65/127] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.321212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.351281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeb719f-6d4a-4382-b0b7-acd4350c5495 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.398840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.399795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.400333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.400644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.401374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:55.401673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.403301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.412414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 63/128] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.413221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.465178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd335a-a909-458a-92da-4dac3217f139 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.506643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.507578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.508036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.508586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.510483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.512194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.514688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 66/129] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.516322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.522859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:55.535559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:55.553404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d0b81-0f53-43e2-8dd1-b46ccd5ec641 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.631602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.632213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ce6de-68c2-44f4-b29e-d58f3454cbfb None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.632668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.632668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.632794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.633896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.636117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.637285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 64/130] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.644833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.702700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.703335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.703618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.704126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.707233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.709556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.715580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 67/131] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.716328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.838583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e693ee1-054d-49bd-bc4e-7f281867b7b0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.864268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:55.900945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:55.900945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:55.900945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:55.900945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:55.902095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:55.903725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:55.905586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 65/132] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:55.906754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:55.911643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33eab7-39ea-4ce8-a62a-f103f27d4731 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:55.938484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:56.010666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:56.011524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.011871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.012214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] PATH_INFO: `/v3/projects/2d0db923f3214483b6ea510ef5d735ea` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.013583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0db923f3214483b6ea510ef5d735ea)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:56.015236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:56.017364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 68/133] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/projects/2d0db923f3214483b6ea510ef5d735ea => generated 312 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:56.018078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:56.028665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201aa38-cce5-466c-a9e0-dd0e8151df4f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:56.068089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:56.068939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.069227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.069505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.071291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:56.073758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:56.075885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 66/134] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:55 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:56.077404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:56.147134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e46c49-9378-4c35-83f6-0b9620cf73c0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:56.182084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1063e-7e82-47b0-9f82-775f7b7ac1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:56.182858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1063e-7e82-47b0-9f82-775f7b7ac1ec None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.183201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1063e-7e82-47b0-9f82-775f7b7ac1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.183544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1063e-7e82-47b0-9f82-775f7b7ac1ec None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573/users/eb0ba9d6a6bf45098fbc94fef46dc27d/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.184054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:56.185618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1063e-7e82-47b0-9f82-775f7b7ac1ec None admin] RBAC: Authorizing `identity:create_grant(project_id=595b8e1093aa49399a51e159588da573, user_id=eb0ba9d6a6bf45098fbc94fef46dc27d, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:56.187180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1063e-7e82-47b0-9f82-775f7b7ac1ec None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:56.202547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ecaa6-e2b7-41f3-a21a-6b564176c25c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:56.213483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 69/135] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:46:56 2022] PUT /identity/v3/projects/595b8e1093aa49399a51e159588da573/users/eb0ba9d6a6bf45098fbc94fef46dc27d/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:46:56.215710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40581142-8dd2-4e8c-822f-990816fda136 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.216657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40581142-8dd2-4e8c-822f-990816fda136 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.217532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40581142-8dd2-4e8c-822f-990816fda136 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.219895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 70/136] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:56.221128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40581142-8dd2-4e8c-822f-990816fda136 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:56.243587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:56.271089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625a37b-d28f-4ce8-b26e-3e5c9a247a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:56.271733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625a37b-d28f-4ce8-b26e-3e5c9a247a70 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.272010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625a37b-d28f-4ce8-b26e-3e5c9a247a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.272310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625a37b-d28f-4ce8-b26e-3e5c9a247a70 None admin] PATH_INFO: `/v3/projects/2d0db923f3214483b6ea510ef5d735ea/users/c6c71c8a128a4b5f93b4e8d9cf8f37da/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.276060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625a37b-d28f-4ce8-b26e-3e5c9a247a70 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d0db923f3214483b6ea510ef5d735ea, user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:56.277584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625a37b-d28f-4ce8-b26e-3e5c9a247a70 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:56.302899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 67/137] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:46:56 2022] PUT /identity/v3/projects/2d0db923f3214483b6ea510ef5d735ea/users/c6c71c8a128a4b5f93b4e8d9cf8f37da/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:46:56.305713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb5459-3270-4e53-8282-5cbdf60b9604 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.306021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb5459-3270-4e53-8282-5cbdf60b9604 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.306303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb5459-3270-4e53-8282-5cbdf60b9604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.341401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b3eb5459-3270-4e53-8282-5cbdf60b9604 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:56.342683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b3eb5459-3270-4e53-8282-5cbdf60b9604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:56.496395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 68/138] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:56.499959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40581142-8dd2-4e8c-822f-990816fda136 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:56.563172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7517f26-c47c-49bb-9f01-89feb9ac5522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:56.563836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517f26-c47c-49bb-9f01-89feb9ac5522 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.564126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517f26-c47c-49bb-9f01-89feb9ac5522 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.564404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7517f26-c47c-49bb-9f01-89feb9ac5522 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573/users/c6c71c8a128a4b5f93b4e8d9cf8f37da/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.567316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517f26-c47c-49bb-9f01-89feb9ac5522 None admin] RBAC: Authorizing `identity:create_grant(project_id=595b8e1093aa49399a51e159588da573, user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:56.568973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7517f26-c47c-49bb-9f01-89feb9ac5522 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:56.587053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 71/139] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:46:56 2022] PUT /identity/v3/projects/595b8e1093aa49399a51e159588da573/users/c6c71c8a128a4b5f93b4e8d9cf8f37da/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:46:56.613367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71968f3-2123-42d3-ac7b-0f868e2a32e7 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.613900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71968f3-2123-42d3-ac7b-0f868e2a32e7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.613900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71968f3-2123-42d3-ac7b-0f868e2a32e7 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.615577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 69/140] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:56.622439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:56.623004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:56.623704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:56.646044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:56.648165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:56.722298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:56.728750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 72/141] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:56.897990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71968f3-2123-42d3-ac7b-0f868e2a32e7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:57.000130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:57.032062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71968f3-2123-42d3-ac7b-0f868e2a32e7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:57.059443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:57.070132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:57.071091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:57.071607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:57.072074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:57.074029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:57.075668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:57.077865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 70/142] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:46:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:57.083928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:57.240777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cca8b-9424-4962-b307-d3abb1eab681 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:57.246559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbf597-0ddf-4ec8-b1d1-d7f525cc4402 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:57.310971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:57.311944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:57.312062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:57.312548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:57.313043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:57.313292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:57.313753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:57.314234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:57.315111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:57.316778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:57.355405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:57.360990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:57.362396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:57.364307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 73/143] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:46:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:46:57.368478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:57.374334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:57.383205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 71/144] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:46:57 2022] POST /identity/v3/users => generated 283 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:46:57.474404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:57.535351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343bdfa-2fe0-499f-89d0-b6dd2717e1b0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:57.577238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:57.577864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:57.578176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:57.578450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:57.579135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:57.579436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:57.580856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:57.590619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 74/145] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:46:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:57.596430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:57.762978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7e771-fb14-4762-991b-13720dbddf42 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:57.804450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:57.805015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:57.805362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:57.805641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] PATH_INFO: `/v3/users/961970d1d0c9409dad8d940af31d9ef4` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:57.806886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] RBAC: Authorizing `identity:get_user(user_id=961970d1d0c9409dad8d940af31d9ef4)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:57.808356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:57.810377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 72/146] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:57 2022] GET /identity/v3/users/961970d1d0c9409dad8d940af31d9ef4 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:57.815701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:57.948803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc7296-0d9f-4ac7-9379-f280cff6beb9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:57.972756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:57.996102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d40ba2-e256-4b33-94d7-91548c35db21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:57.996749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40ba2-e256-4b33-94d7-91548c35db21 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:57.997040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40ba2-e256-4b33-94d7-91548c35db21 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:57.997298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40ba2-e256-4b33-94d7-91548c35db21 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:57.998712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d40ba2-e256-4b33-94d7-91548c35db21 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:58.000172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d40ba2-e256-4b33-94d7-91548c35db21 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:58.002099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 75/147] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:46:57 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:58.008044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:58.113819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:58.161018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d500ce-0a76-4d34-a965-21ca71116d41 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:58.203242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d621-a2fe-4ce0-ac9b-f1d1bf7193d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:58.203959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d621-a2fe-4ce0-ac9b-f1d1bf7193d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:58.204236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d621-a2fe-4ce0-ac9b-f1d1bf7193d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:58.204561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d621-a2fe-4ce0-ac9b-f1d1bf7193d5 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573/users/961970d1d0c9409dad8d940af31d9ef4/roles/e99d1aaf2c754f72a1b1f8c6bba548e1` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:58.206958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1d621-a2fe-4ce0-ac9b-f1d1bf7193d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=595b8e1093aa49399a51e159588da573, user_id=961970d1d0c9409dad8d940af31d9ef4, role_id=e99d1aaf2c754f72a1b1f8c6bba548e1)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:58.208422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1d621-a2fe-4ce0-ac9b-f1d1bf7193d5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:58.223103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 73/148] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:46:58 2022] PUT /identity/v3/projects/595b8e1093aa49399a51e159588da573/users/961970d1d0c9409dad8d940af31d9ef4/roles/e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:46:58.456358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05f621-ea50-4152-9fc2-d0f00621046c None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:58.457011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05f621-ea50-4152-9fc2-d0f00621046c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:58.457501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05f621-ea50-4152-9fc2-d0f00621046c None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:58.459143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 76/149] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:58.465561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:58.465850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:58.466159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:58.491070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:58.492282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:58.582728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 74/150] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:46:58.937359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05f621-ea50-4152-9fc2-d0f00621046c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:59.048791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05f621-ea50-4152-9fc2-d0f00621046c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:59.089000 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:59.090030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:59.090514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:59.090983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:59.107836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:46:59.113006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:46:59.113869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:46:59.114684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:46:59.117257 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 77/151] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:46:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:46:59.123153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:59.202524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:59.256961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa62665-a53c-490d-91c8-cfbd4a3c1765 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:59.305760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:59.306407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:59.306733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:59.307059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:59.307802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:46:59.308108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:59.309555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:59.323488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 75/152] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:46:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:59.329535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:59.410182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:59.487169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ad936-7bcd-49d8-a6ad-3006ef9d4357 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:59.528561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06498d3-daf2-48dd-a847-64cd88d27743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:59.529201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06498d3-daf2-48dd-a847-64cd88d27743 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:59.529483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06498d3-daf2-48dd-a847-64cd88d27743 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:59.529755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06498d3-daf2-48dd-a847-64cd88d27743 None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:59.530906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06498d3-daf2-48dd-a847-64cd88d27743 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:59.532344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06498d3-daf2-48dd-a847-64cd88d27743 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:46:59.534141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 78/153] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:46:59 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:46:59.543795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:46:59.616392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7379461-920f-4c5d-97fe-5b29c9dd1c4f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:59.616745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7379461-920f-4c5d-97fe-5b29c9dd1c4f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:59.616943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7379461-920f-4c5d-97fe-5b29c9dd1c4f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:59.619757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 79/154] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:46:59.624112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:59.624374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:59.624647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:59.658415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:46:59.661951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:46:59.707459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d3873-2337-442c-8156-22faef8f03ae None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:46:59.742609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:46:59.767405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 80/155] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:46:59.783439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0019e-ce6b-4d64-be60-7fdc57cd4fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:46:59.784071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0019e-ce6b-4d64-be60-7fdc57cd4fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:46:59.784393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0019e-ce6b-4d64-be60-7fdc57cd4fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:46:59.784635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0019e-ce6b-4d64-be60-7fdc57cd4fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:46:59.896610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0019e-ce6b-4d64-be60-7fdc57cd4fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:46:59.898113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0019e-ce6b-4d64-be60-7fdc57cd4fe6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:00.022378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 76/156] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:46:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:00.029138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:00.061653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08948e2-3823-465f-b98f-bcc4c5e2db70 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.061977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08948e2-3823-465f-b98f-bcc4c5e2db70 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.062528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08948e2-3823-465f-b98f-bcc4c5e2db70 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.063887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 77/157] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:00.076072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b6f49-202f-43c2-8141-5128d310231a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.076365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b6f49-202f-43c2-8141-5128d310231a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.076749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b6f49-202f-43c2-8141-5128d310231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.107079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-603b6f49-202f-43c2-8141-5128d310231a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:00.108280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-603b6f49-202f-43c2-8141-5128d310231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:00.114503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:00.136609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:00.248705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 78/158] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:00.249103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e27bf7-9637-4a11-93fa-43c306e9a2bd None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:00.307329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204aff88-47ee-4345-a95d-8b7d89517940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:00.307898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aff88-47ee-4345-a95d-8b7d89517940 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.308177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aff88-47ee-4345-a95d-8b7d89517940 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.308452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204aff88-47ee-4345-a95d-8b7d89517940 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.309604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204aff88-47ee-4345-a95d-8b7d89517940 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:00.311082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204aff88-47ee-4345-a95d-8b7d89517940 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:00.313013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 81/159] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:00 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:00.318988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b6f49-202f-43c2-8141-5128d310231a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:00.439608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b757c-780b-4a6c-a4d2-11385deb4c8f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.439928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b757c-780b-4a6c-a4d2-11385deb4c8f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.440151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b757c-780b-4a6c-a4d2-11385deb4c8f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.441499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 82/160] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:00.452457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.452731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.452996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.479962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:00.481160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:00.490736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b6f49-202f-43c2-8141-5128d310231a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:00.539834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:00.540530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.540807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.541635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.542449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] RBAC: Adding query filter params (role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:00.542865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:00.544429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:00.550792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 79/161] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:00 2022] GET /identity/v3/role_assignments?user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=399f7b70c70b4f09addcf156bf604eab => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:00.551594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:00.569095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:00.624388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 83/162] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:00.699374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a25c78-5d54-4ef2-98b9-ed9e7842f466 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:00.737570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:00.738251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.738523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.738837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.750815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:00.756418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:00.757361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:00.758885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 80/163] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:00.763711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:00.819327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:00.883193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13262378-3b59-4edc-83a7-ed7891937034 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:00.928142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:00.929128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:00.929574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:00.930024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:00.930875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:00.932200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:00.933744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:00.942785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 84/164] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:00.948814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:00.967568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:01.099732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:01.156931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33d36a-a5c0-4a78-8858-3a8947366437 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:01.176773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ddafc-77d6-4ec0-8263-0dca3f8e04d1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:01.188408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:01.217793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:01.218814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:01.219165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:01.219513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] PATH_INFO: `/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:01.221335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=eb0ba9d6a6bf45098fbc94fef46dc27d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:01.223034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:01.226858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 81/165] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:00 2022] GET /identity/v3/users/eb0ba9d6a6bf45098fbc94fef46dc27d => generated 247 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:01.230137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:01.230783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:01.231063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:01.231338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:01.232968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:01.258676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:01.266349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:01.267226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:01.268252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 85/166] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:01.274079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:01.407850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326c2df-6b07-4f5f-b978-504ffbc9a3b0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:01.420204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7cb87-1103-4fe7-8b77-4c0b48417ae5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:01.444700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:01.445375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:01.445657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:01.445928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:01.446648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:01.446945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:01.448860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:01.457976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 86/167] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:01.458912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:01.488705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:01.502687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:01.503305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:01.503585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:01.503862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:01.524897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:01.564232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262635-e7fc-4d52-9c2f-35f8ac4c72e3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:01.600284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:01.601245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:01.601698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:01.602183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:01.612871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:01.617986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:01.618945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:01.620281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:01.622645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 87/168] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:01.623979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:01.639444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:01.642199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:01.793944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:01.817305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d9d5d-02a0-4244-9d23-5d826162d770 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:01.847789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 82/169] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:47:01.848731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:01.898207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:01.903782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:01.906254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:01.906254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:01.906254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:01.907442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:01.908904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:01.910521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 88/170] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:01 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:01.911289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:02.009636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd89cd2-c2e3-4b30-b512-7b1ed2188441 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:02.079149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:02.080079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.080580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.081033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.081980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:02.082746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:02.084323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:02.093591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 83/171] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:02.094763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:02.106748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a983b-a8ee-4ffb-b460-93931957046a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:02.167301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:02.170025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.170025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.170025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.170025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:02.171031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:02.172949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 89/172] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:01 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:02.173774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:02.205152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:02.262095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572b842-b949-4fa0-8b5d-15829d96d371 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:02.266484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:02.334705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cea9e6-090a-4f3c-9daa-93f5396f8b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:02.335221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea9e6-090a-4f3c-9daa-93f5396f8b3c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.335494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea9e6-090a-4f3c-9daa-93f5396f8b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.336432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cea9e6-090a-4f3c-9daa-93f5396f8b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.347613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa54a5-260b-4d8d-8712-57438f4013c9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:02.407423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:02.408114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.408390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.408661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] PATH_INFO: `/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.410767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] RBAC: Authorizing `identity:get_user(user_id=c6c71c8a128a4b5f93b4e8d9cf8f37da)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:02.412218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:02.413809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 90/173] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:02 2022] GET /identity/v3/users/c6c71c8a128a4b5f93b4e8d9cf8f37da => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:02.414664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:02.463620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea9e6-090a-4f3c-9daa-93f5396f8b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:02.465799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cea9e6-090a-4f3c-9daa-93f5396f8b3c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:02.596216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 84/174] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:02.597711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39bf0c-18e2-4f1d-9c2d-267650989525 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.598014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39bf0c-18e2-4f1d-9c2d-267650989525 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.598406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39bf0c-18e2-4f1d-9c2d-267650989525 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.599675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 85/175] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:02.606019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39bf0c-18e2-4f1d-9c2d-267650989525 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:02.606019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:02.639384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:02.679962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa5c99-7393-4a88-8c00-b0a156340955 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:02.743174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:02.744238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.745181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.745586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.746466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=eb0ba9d6a6bf45098fbc94fef46dc27d) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:02.746922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:02.748946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3c289-2fac-4504-888b-0b8664fdd5f9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:02.755640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 91/176] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:02 2022] GET /identity/v3/role_assignments?user.id=eb0ba9d6a6bf45098fbc94fef46dc27d&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:02.756251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39bf0c-18e2-4f1d-9c2d-267650989525 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:02.757141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53253049-de13-4564-918f-a47939bc871c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.757579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53253049-de13-4564-918f-a47939bc871c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.758013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53253049-de13-4564-918f-a47939bc871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.778462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-53253049-de13-4564-918f-a47939bc871c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:02.780013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-53253049-de13-4564-918f-a47939bc871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:02.817788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47138b48-2f0b-4abf-a2ec-2a173d9cd86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:02.818949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47138b48-2f0b-4abf-a2ec-2a173d9cd86e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:02.819543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47138b48-2f0b-4abf-a2ec-2a173d9cd86e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:02.820139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47138b48-2f0b-4abf-a2ec-2a173d9cd86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:02.858457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 92/177] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:02.859676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53253049-de13-4564-918f-a47939bc871c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:02.956797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47138b48-2f0b-4abf-a2ec-2a173d9cd86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:02.958346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47138b48-2f0b-4abf-a2ec-2a173d9cd86e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:02.961361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:02.991250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.012608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53253049-de13-4564-918f-a47939bc871c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.060521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f214f-db4f-4a19-8be8-e5e06a201269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.061426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f214f-db4f-4a19-8be8-e5e06a201269 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.061726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f214f-db4f-4a19-8be8-e5e06a201269 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.062393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f214f-db4f-4a19-8be8-e5e06a201269 None admin] PATH_INFO: `/v3/projects/2d0db923f3214483b6ea510ef5d735ea` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.063550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f214f-db4f-4a19-8be8-e5e06a201269 None admin] RBAC: Authorizing `identity:get_project(project_id=2d0db923f3214483b6ea510ef5d735ea)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:03.064998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f214f-db4f-4a19-8be8-e5e06a201269 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:03.068287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 93/178] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:02 2022] GET /identity/v3/projects/2d0db923f3214483b6ea510ef5d735ea => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:03.069914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605d3d-7867-4954-878a-d71bd0985fe9 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.070284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605d3d-7867-4954-878a-d71bd0985fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.070652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c605d3d-7867-4954-878a-d71bd0985fe9 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.072042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 94/179] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:03.073915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262ea0-2335-4a98-bd82-3b85fa055c50 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.074224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262ea0-2335-4a98-bd82-3b85fa055c50 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.074495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c262ea0-2335-4a98-bd82-3b85fa055c50 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.075812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 95/180] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:03.076903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa51216-590e-4636-9b3d-f38157e66c6f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.077186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa51216-590e-4636-9b3d-f38157e66c6f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.077456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa51216-590e-4636-9b3d-f38157e66c6f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.078952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 96/181] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:03.082030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa51216-590e-4636-9b3d-f38157e66c6f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.091149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 86/182] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:03.092756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d5aab-6820-40ea-87f4-6363c8bcef3e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.093045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d5aab-6820-40ea-87f4-6363c8bcef3e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.093317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d5aab-6820-40ea-87f4-6363c8bcef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.115361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-9a1d5aab-6820-40ea-87f4-6363c8bcef3e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:03.116407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9a1d5aab-6820-40ea-87f4-6363c8bcef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:03.194845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 87/183] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:03.196139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.196422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.196688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.201900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa51216-590e-4636-9b3d-f38157e66c6f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.215414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:03.216868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:03.237093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.237793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.238165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.238437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.239212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, scope.project.id=2d0db923f3214483b6ea510ef5d735ea, user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:03.239453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:03.240883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9c32c1-5110-40e6-b730-dd7d8f308e2d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:03.244577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.254021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 97/184] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/role_assignments?user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da&scope.project.id=2d0db923f3214483b6ea510ef5d735ea&role.id=6768099ea599447f9de57814f623441c => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:03.254021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.254021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.254021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.270828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:03.272199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:03.292091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.298557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 88/185] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:03.307603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.404991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 98/186] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:03.407851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.500959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83155b1d-71b0-4bad-b90b-8a050150b922 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.549341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.550393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.550889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.551458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.552770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:03.554545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:03.556506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 89/187] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:03.557775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.569351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0afc49-8952-4f9c-96c9-e9424616a7f8 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.595920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.614102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.615297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.615796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.616603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.627717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:03.632543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:03.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:03.634240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:03.635379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 99/188] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:03.636372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.649035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.687350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa33e-5c31-4118-ac3b-f12cc82f39dd None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.725639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.726297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.726571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.726835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.738144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:03.742932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:03.743785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:03.745264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 90/189] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:03.745944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.760839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f609c6b-dee5-41c2-8047-9881956f1e20 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.794692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.795291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.795566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.795838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.796556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:03.796859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:03.798352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:03.804168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 100/190] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/role_assignments?user.id=c6c71c8a128a4b5f93b4e8d9cf8f37da&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=6768099ea599447f9de57814f623441c => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:03.804930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.839220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.849725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0e839-42cf-4733-abd8-5ffa01cc3f83 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.887182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.887823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.888100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.888366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.890791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:03.901433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:03.906474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:03.907366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:03.908370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 91/191] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:03.909521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:03.933519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0c7f2-50af-4381-9fa7-d2596a0dcb42 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:03.971642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:03.972263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:03.972568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:03.972840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:03.973506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:03.973806 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:03.975223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:03.985149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 101/192] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:03.986063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.023389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d8d42-2bd2-4115-bf4e-36d29bc489a7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.065341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.066129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.066408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.066679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.067350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:04.067644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.069025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.077238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 92/193] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:04.078310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.093587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.099581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd331f80-277b-4137-8fd8-ad144e478ca1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.132746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.133369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.133643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.133912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.135074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.136495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.138201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 102/194] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:04.138802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.141169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.181416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e0cfc-c658-4c8e-b6e6-c11b86522afb None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.216257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.216857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.217129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.217394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.218082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:04.218379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.219730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.227621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 93/195] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:04.228124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.238466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd56a8-919f-45e0-95a8-1a9e489c10e6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.271992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.272610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.272890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.273161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] PATH_INFO: `/v3/users/fd3faae32a6046c5acf1bbad810329f6` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.285812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] RBAC: Authorizing `identity:get_user(user_id=fd3faae32a6046c5acf1bbad810329f6)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.287275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.288836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 103/196] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/users/fd3faae32a6046c5acf1bbad810329f6 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:04.290825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.303621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.326406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86151b-c978-4ed1-87a0-88005dd6c075 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.352588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.360506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.361133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.361434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.361685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] PATH_INFO: `/v3/users/961970d1d0c9409dad8d940af31d9ef4` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.362840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] RBAC: Authorizing `identity:get_user(user_id=961970d1d0c9409dad8d940af31d9ef4)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.364271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.365833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 94/197] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/users/961970d1d0c9409dad8d940af31d9ef4 => generated 289 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:04.366693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.397393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b28f03-2ec2-4d34-9771-c6a407f2ae6c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.430333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.430937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.431219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.431494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.432710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.434166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.453783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 104/198] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:04 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:04.454622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.464050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b2551f-332d-4213-b2e3-8808758b021c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.506267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.507130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.507408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.508135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] PATH_INFO: `/v3/users/fd3faae32a6046c5acf1bbad810329f6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.509495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3faae32a6046c5acf1bbad810329f6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.511023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.512907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 95/199] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/users/fd3faae32a6046c5acf1bbad810329f6 => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:04.513544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.529867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.570321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a4517-1e95-4fd9-8471-a0860a0f553e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.590317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.611518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.612151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.612428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.612698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.637115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f381ac-4ffb-4dc1-bb9d-8e0f88ec5503 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.676037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.676756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.677029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.677293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:04.701467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.702976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.759054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:04.761873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:04.780524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.829364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 105/200] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:04.830240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.849767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:04.889895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 96/201] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:04.890572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:04.953555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e86fca-7092-4853-9e96-6771a530f658 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:04.991062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:04.991668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:04.991943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:04.992213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.007535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9413a-d2ef-4d60-a433-31871a3f78b7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:05.023211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:05.048053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:05.049100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:05.049577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:05.050107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.051560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.053237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.055392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 97/202] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:05.056529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:05.068334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.070177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.094643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:05.173695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4602c55-c7d7-45f9-8b6b-51bbb7523569 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:05.207705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:05.208419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 106/203] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:05.209025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:05.209254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:05.209790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:05.210320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.211767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.213851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.216218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 98/204] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:05 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:05.217335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:05.282435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:05.319092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f502963-730b-4e0e-a50e-d0fc01f26df0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:05.343711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:05.355122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45346f8b-033c-4da2-9869-2739e056c00f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:05.373480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:05.374959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:05.375808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:05.376272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.377204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=fd3faae32a6046c5acf1bbad810329f6) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:05.377787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.380293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.389280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 107/205] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:05 2022] GET /identity/v3/role_assignments?user.id=fd3faae32a6046c5acf1bbad810329f6&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:05.390441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:05.393394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:05.393927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:05.394250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:05.394524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.396045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.397229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.399384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 99/206] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:05 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:05.405503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:05.516950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f277a-51ec-450f-98c3-c50bf7f1a3dd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:05.547804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74517867-52d7-4e16-8e72-9742de95ca7a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:05.554198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:05.554951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:05.555384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:05.555810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.557508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] RBAC: Adding query filter params (role.id=e99d1aaf2c754f72a1b1f8c6bba548e1, scope.project.id=595b8e1093aa49399a51e159588da573, user.id=961970d1d0c9409dad8d940af31d9ef4) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:05.557690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.559425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb0c0f-8455-4802-ba04-21d844fbc8b9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.566901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 108/207] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:05 2022] GET /identity/v3/role_assignments?user.id=961970d1d0c9409dad8d940af31d9ef4&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:05.568195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:05.589902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:05.590806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:05.591258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:05.591669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:05.592611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] RBAC: Adding query filter params (role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=fd3faae32a6046c5acf1bbad810329f6) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:05.593067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:05.594778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8157155-9363-448d-b91a-725fa467cfbe None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:05.601582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 100/208] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:05 2022] GET /identity/v3/role_assignments?user.id=fd3faae32a6046c5acf1bbad810329f6&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=399f7b70c70b4f09addcf156bf604eab => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:06.824187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac27b79-f674-4f5c-bf55-362f4b536d1e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:06.825170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac27b79-f674-4f5c-bf55-362f4b536d1e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:06.825664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac27b79-f674-4f5c-bf55-362f4b536d1e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:06.827316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 109/209] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:06.833047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c4c5a-6dac-45cd-8a7c-f27440b471ec None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:06.833820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c4c5a-6dac-45cd-8a7c-f27440b471ec None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:06.834309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c4c5a-6dac-45cd-8a7c-f27440b471ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:06.852896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-497c4c5a-6dac-45cd-8a7c-f27440b471ec None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:06.854651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-497c4c5a-6dac-45cd-8a7c-f27440b471ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:06.924827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 101/210] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:07.038033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce66a4-0fdf-4bcf-8858-66b7b98271c0 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.038387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce66a4-0fdf-4bcf-8858-66b7b98271c0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.038582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce66a4-0fdf-4bcf-8858-66b7b98271c0 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.040062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 110/211] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:07.050067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.050357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.050628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.068943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:07.070242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:07.141104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 102/212] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:07.203264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce66a4-0fdf-4bcf-8858-66b7b98271c0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:07.251997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:07.308493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce66a4-0fdf-4bcf-8858-66b7b98271c0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:07.343901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:07.344917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.345395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.345864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.357461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:07.363463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:07.364333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:07.365117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:07.367411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 111/213] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:07.375886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:07.414699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:07.429600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:07.477467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a69a3-bc2c-4d89-8752-030a1c79c8ec None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:07.520974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:07.521735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.522145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.522516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.523288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:07.523689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:07.525177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:07.534394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 103/214] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:07.540156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:07.560848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ea08e-9b20-49cf-8778-662aceac7298 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:07.564214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:07.611828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:07.612738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.613487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.613934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.616269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:07.617942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:07.621831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 112/215] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:07.634747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:07.675407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60d1ce-8fda-43b2-b31d-8ed47f734edc None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:07.677201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:07.723610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85daa325-c061-43a4-aaa1-630c06240bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:07.724800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85daa325-c061-43a4-aaa1-630c06240bdd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.725077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85daa325-c061-43a4-aaa1-630c06240bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.725345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85daa325-c061-43a4-aaa1-630c06240bdd None admin] PATH_INFO: `/v3/users/2678daf5b93c47d48cd48802e3484e00` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.740213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85daa325-c061-43a4-aaa1-630c06240bdd None admin] RBAC: Authorizing `identity:get_user(user_id=2678daf5b93c47d48cd48802e3484e00)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:07.741599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85daa325-c061-43a4-aaa1-630c06240bdd None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:07.743405 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 104/216] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:07 2022] GET /identity/v3/users/2678daf5b93c47d48cd48802e3484e00 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:07.744904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0af36c-ee71-417f-bbcd-36d670eccb37 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.745182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0af36c-ee71-417f-bbcd-36d670eccb37 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.745447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0af36c-ee71-417f-bbcd-36d670eccb37 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.746912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 105/217] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:07.750042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0af36c-ee71-417f-bbcd-36d670eccb37 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:07.762935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef0619-d5af-4f99-afa4-e7e0b1fccc55 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:07.805220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff8add-3189-4777-a332-863b35ed90d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:07.805952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff8add-3189-4777-a332-863b35ed90d8 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.806309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff8add-3189-4777-a332-863b35ed90d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.807223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff8add-3189-4777-a332-863b35ed90d8 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.808038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff8add-3189-4777-a332-863b35ed90d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:07.809504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff8add-3189-4777-a332-863b35ed90d8 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:07.846312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 113/218] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:07 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:47:07.846312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf39b-a040-46ba-b562-4656066d9682 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.846312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf39b-a040-46ba-b562-4656066d9682 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.846312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcf39b-a040-46ba-b562-4656066d9682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.856531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0af36c-ee71-417f-bbcd-36d670eccb37 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:07.874498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6adcf39b-a040-46ba-b562-4656066d9682 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:07.877352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6adcf39b-a040-46ba-b562-4656066d9682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:07.901508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:07.902280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:07.902673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:07.903071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:07.912022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:07.943918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:07.973804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 114/219] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:07.993590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:07.995512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:08.010662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb219-6679-4fa5-830a-563550c099f1 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.010959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb219-6679-4fa5-830a-563550c099f1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.011617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bb219-6679-4fa5-830a-563550c099f1 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.013556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 115/220] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:08.019516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116f253-db53-4392-b077-1fb22c11815b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.019792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116f253-db53-4392-b077-1fb22c11815b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.020067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116f253-db53-4392-b077-1fb22c11815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.045077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7116f253-db53-4392-b077-1fb22c11815b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:08.046285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7116f253-db53-4392-b077-1fb22c11815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:08.115823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 106/221] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:47:08.120598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.145575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 116/222] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:08.192315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:08.232962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71176c22-bad4-471f-9ed7-c7cc268792b6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.242883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116f253-db53-4392-b077-1fb22c11815b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.271539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.272403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.272696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.272960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.274133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:08.276075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:08.278342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 107/223] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:08.285895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.358275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:08.374875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116f253-db53-4392-b077-1fb22c11815b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.399888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaf239-e927-4893-aea2-2afca0f6cb2b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.419791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.420710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.421189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.421646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.432618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:08.437932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:08.444718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:08.445619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 117/224] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:08.446468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.449352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.449551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.449784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.450094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.450831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=2678daf5b93c47d48cd48802e3484e00) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:08.451134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:08.452567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:08.462213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 108/225] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/role_assignments?user.id=2678daf5b93c47d48cd48802e3484e00&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=6768099ea599447f9de57814f623441c => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:08.463575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.612063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c16de5-e15a-4139-967c-6723d6c9126e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.623188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae4573-af08-423f-9ac9-51dce0c37f8a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.684475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.685116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.685425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.685650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.686356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:08.687340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:08.688074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:08.691117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.691995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.692528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.692992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.698949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 109/226] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:08.704523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.710456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:08.717059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:08.717944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:08.718860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 118/227] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:08.723957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.747807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:08.823837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:08.887014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0795b-ab01-4319-9e9a-f48732bed9ce None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.937371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.938034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.938316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.938587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] PATH_INFO: `/v3/users/fd3faae32a6046c5acf1bbad810329f6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.939709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3faae32a6046c5acf1bbad810329f6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:08.941148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:08.942626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d396e0f-72f1-4b20-9f2c-e23ec5e1cfca None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:08.943075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 110/228] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/users/fd3faae32a6046c5acf1bbad810329f6 => generated 283 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:08.954973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:08.988137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:08.988760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:08.989036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:08.989308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:08.990323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:08.990625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:08.992021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:09.020373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 119/229] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:09.022342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:09.096571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:09.097066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a52a8-0f11-4796-bd31-c81457bc00b3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:09.121416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:09.131718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a0cbc-e63e-41c1-9e52-7ac7891a1480 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:09.134637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:09.135398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:09.135674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:09.135985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:09.137254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:09.138673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:09.140632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 111/230] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:08 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:09.146980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:09.167171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:09.167813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:09.168089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:09.168357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] PATH_INFO: `/v3/users/fd3faae32a6046c5acf1bbad810329f6` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:09.169509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3faae32a6046c5acf1bbad810329f6)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:09.170971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:09.172549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 120/231] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:09 2022] GET /identity/v3/users/fd3faae32a6046c5acf1bbad810329f6 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:09.177878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:09.297905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25d271-0e51-492b-a4f0-25fc1d295110 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:09.312892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb4ee7e-1083-48d2-9591-644395330594 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:09.337436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:09.339395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:09.339676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:09.339951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:09.341255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:09.344220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:09.346794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 121/232] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:09 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:09.353059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:09.376743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:09.377137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dba8d1-fa4e-4764-8e0e-459c91c6f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:09.378408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dba8d1-fa4e-4764-8e0e-459c91c6f4f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:09.378832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dba8d1-fa4e-4764-8e0e-459c91c6f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:09.379115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dba8d1-fa4e-4764-8e0e-459c91c6f4f3 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/fd3faae32a6046c5acf1bbad810329f6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:09.382028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dba8d1-fa4e-4764-8e0e-459c91c6f4f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1259daa670c46fcacf1d9b1449f2537, user_id=fd3faae32a6046c5acf1bbad810329f6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:09.384102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dba8d1-fa4e-4764-8e0e-459c91c6f4f3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:09.399645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 112/233] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:09 2022] PUT /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/fd3faae32a6046c5acf1bbad810329f6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:09.543115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01452d-4449-480c-bfda-d86d3309e5e8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:09.591714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88576d57-9688-4053-9522-87e16350022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:09.592393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576d57-9688-4053-9522-87e16350022e None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:09.592672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576d57-9688-4053-9522-87e16350022e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:09.592942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88576d57-9688-4053-9522-87e16350022e None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/fd3faae32a6046c5acf1bbad810329f6/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:09.595120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88576d57-9688-4053-9522-87e16350022e None admin] RBAC: Authorizing `identity:create_grant(project_id=a1259daa670c46fcacf1d9b1449f2537, user_id=fd3faae32a6046c5acf1bbad810329f6, role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:09.596626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88576d57-9688-4053-9522-87e16350022e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:09.622397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 122/234] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:09 2022] PUT /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/fd3faae32a6046c5acf1bbad810329f6/roles/399f7b70c70b4f09addcf156bf604eab => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:10.753590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a452f-e170-4ac2-a587-ade50177a7ad None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:10.754115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a452f-e170-4ac2-a587-ade50177a7ad None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:10.754221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a452f-e170-4ac2-a587-ade50177a7ad None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:10.755784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 113/235] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:10.761333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7855b1f-990c-4057-afc1-f8d28e47d313 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:10.762320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7855b1f-990c-4057-afc1-f8d28e47d313 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:10.762799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7855b1f-990c-4057-afc1-f8d28e47d313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:10.787680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f7855b1f-990c-4057-afc1-f8d28e47d313 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:10.789253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f7855b1f-990c-4057-afc1-f8d28e47d313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:10.874424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:10.886298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 123/236] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:10.902956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c118ab-336f-4c24-b6f4-da69916b1e91 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:10.903256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c118ab-336f-4c24-b6f4-da69916b1e91 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:10.903509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c118ab-336f-4c24-b6f4-da69916b1e91 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:10.904996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 114/237] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:10.910418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09acd1-a62a-4e84-98aa-1159ccb1fcb0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:10.910677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09acd1-a62a-4e84-98aa-1159ccb1fcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:10.910901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09acd1-a62a-4e84-98aa-1159ccb1fcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:10.929496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ea09acd1-a62a-4e84-98aa-1159ccb1fcb0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:10.930754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ea09acd1-a62a-4e84-98aa-1159ccb1fcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:11.013586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 124/238] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:11.099366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bf600-70de-4c32-b447-d508457b9553 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.099670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bf600-70de-4c32-b447-d508457b9553 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.099916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0bf600-70de-4c32-b447-d508457b9553 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.101400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 115/239] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:11.107029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.107319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.107596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.126171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:11.127350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:11.197915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 125/240] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:11.232596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bf600-70de-4c32-b447-d508457b9553 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.286673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:11.353430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0bf600-70de-4c32-b447-d508457b9553 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:11.389723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:11.390415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.390689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.391010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.399210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.416817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:11.428897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:11.429762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:11.430567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 116/241] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:11.437879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.482159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:11.532203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9db7-7551-4625-9a81-e045d7cba5ed None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:11.571753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:11.572978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.573266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.573537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.586041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:11.591609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:11.592459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:11.593224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 126/242] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:11.594556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.617697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1c1ec-76c6-404f-94a9-9775e6ba263b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:11.632423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:11.673117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:11.674066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.674533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.675502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.677364 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:11.677866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:11.680322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:11.696521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 117/243] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:11.699017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.725743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604088-6dd9-4db5-8b71-ce09b6ee0d84 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:11.745785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:11.772514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:11.774238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.774993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.775962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.777602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:11.780285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:11.783091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 127/244] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:11.787813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.826985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893f765-7d75-4968-9a66-7f9fa0273322 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:11.863820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:11.864239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.864518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.864836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.865515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:11.865813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:11.867228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:11.881011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 118/245] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:11.881011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:11.915100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:11.935643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184ec1e-650b-4960-9359-460429cffdfd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:11.975970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:11.976642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:11.976921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:11.977195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] PATH_INFO: `/v3/users/ed5b761801074363943fd9f724baa54f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:11.992460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5b761801074363943fd9f724baa54f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:11.993989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:11.995951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 128/246] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/users/ed5b761801074363943fd9f724baa54f => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:11.996509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:12.004696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383322b-fd33-4773-b1a0-2f9f5c5ace8a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:12.049601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:12.051745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:12.052383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.052809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.053223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.054006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:12.055678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:12.086349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 119/247] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:11 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:12.087486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:12.124131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e409d27-4073-4803-83cd-c77a13ebd226 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:12.170097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:12.171226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.172410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.174158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] PATH_INFO: `/v3/users/2678daf5b93c47d48cd48802e3484e00` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.176225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] RBAC: Authorizing `identity:get_user(user_id=2678daf5b93c47d48cd48802e3484e00)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:12.182859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:12.185559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 129/248] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:11 2022] GET /identity/v3/users/2678daf5b93c47d48cd48802e3484e00 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:12.193486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:12.278363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:12.304608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda69557-33f9-4847-bace-a0eb69ace575 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:12.323545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ecb2e-e6e5-4aa6-95ff-822d84141467 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:12.336323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:12.358492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0b495-d330-4ded-b09f-0ae8427ec249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:12.359251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0b495-d330-4ded-b09f-0ae8427ec249 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.359534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0b495-d330-4ded-b09f-0ae8427ec249 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.359838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0b495-d330-4ded-b09f-0ae8427ec249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.363865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33a96e-9669-4951-ab05-e07c34930a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:12.364502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33a96e-9669-4951-ab05-e07c34930a2e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.364786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33a96e-9669-4951-ab05-e07c34930a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.365064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33a96e-9669-4951-ab05-e07c34930a2e None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.366338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33a96e-9669-4951-ab05-e07c34930a2e None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:12.367787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d33a96e-9669-4951-ab05-e07c34930a2e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:12.370149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 130/249] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:12 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:12.371241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f181de-80e3-419b-a6ef-2621b1845192 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.371879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f181de-80e3-419b-a6ef-2621b1845192 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.371879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f181de-80e3-419b-a6ef-2621b1845192 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.375372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 131/250] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:12.375372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1febd-a693-4166-ad1c-1d8bc2294815 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.375372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1febd-a693-4166-ad1c-1d8bc2294815 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.375623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1febd-a693-4166-ad1c-1d8bc2294815 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.376945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 132/251] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:12.377683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1febd-a693-4166-ad1c-1d8bc2294815 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:12.443765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0b495-d330-4ded-b09f-0ae8427ec249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:12.444332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0b495-d330-4ded-b09f-0ae8427ec249 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:12.508369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1febd-a693-4166-ad1c-1d8bc2294815 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:12.550892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185219-052e-4622-b2e1-ace2129e4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:12.551490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185219-052e-4622-b2e1-ace2129e4dcf None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.551775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185219-052e-4622-b2e1-ace2129e4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.552051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185219-052e-4622-b2e1-ace2129e4dcf None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/2678daf5b93c47d48cd48802e3484e00/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.555309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f185219-052e-4622-b2e1-ace2129e4dcf None admin] RBAC: Authorizing `identity:create_grant(project_id=a1259daa670c46fcacf1d9b1449f2537, user_id=2678daf5b93c47d48cd48802e3484e00, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:12.556843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f185219-052e-4622-b2e1-ace2129e4dcf None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:12.558892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:12.571161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 120/252] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:12.573471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.573982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.574469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.586106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 133/253] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:12 2022] PUT /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/2678daf5b93c47d48cd48802e3484e00/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:12.587631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.587950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.588794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.605372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:12.608905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:12.627143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:12.630551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:12.699957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:12.727854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 121/254] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:12.729307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:12.802976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 134/255] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:12.849435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba07b6f-ef5f-40af-ad52-f45c8a71df97 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:12.872806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:12.894684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:12.895659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:12.896126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:12.896585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:12.898913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:12.901133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:12.902986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 122/256] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:12 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:12.908907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.055900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aedd93-7e6d-49a4-bc75-af24a7deb8b5 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:13.091694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:13.092672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:13.093141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:13.093596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:13.094549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] RBAC: Adding query filter params (role.id=e99d1aaf2c754f72a1b1f8c6bba548e1, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=ed5b761801074363943fd9f724baa54f) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:13.095065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:13.096706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:13.102911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 135/257] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:12 2022] GET /identity/v3/role_assignments?user.id=ed5b761801074363943fd9f724baa54f&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:13.123299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.261840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393bb3f-5fe5-448d-83d7-9bc03ac85b58 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:13.309053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:13.309627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:13.309911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:13.310211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:13.323085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:13.328396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:13.330076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:13.331620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 123/258] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:13.339678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.366765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.386889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:13.472199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:13.501504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456627a-9c01-475e-8e5c-a344573e3604 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:13.563852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286041c-d327-4416-b0bc-bb88f7cd72ff None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:13.568546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:13.569166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:13.569444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:13.569712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:13.570408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:13.570702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:13.572068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:13.589846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 136/259] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:13.606482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.638610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:13.639287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:13.639567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:13.639835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:13.653489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:13.662019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:13.662947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:13.663814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 124/260] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:13.679527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.825263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e59ed0-b6dd-410f-b6b6-07d5fdb3c63f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:13.827023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:13.831054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5660f8-12b6-4c70-aac7-143219f25f51 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:13.843155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:13.862721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:13.863523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:13.863925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:13.864302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:13.865159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:13.865581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:13.867117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:13.876593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 125/261] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:13.882190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:13.899043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:13.899687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:13.899969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:13.900248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] PATH_INFO: `/v3/users/fd3faae32a6046c5acf1bbad810329f6` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:13.903350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] RBAC: Authorizing `identity:get_user(user_id=fd3faae32a6046c5acf1bbad810329f6)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:13.904891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:13.910703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 137/262] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/users/fd3faae32a6046c5acf1bbad810329f6 => generated 283 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:13.912409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:14.005334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116033-66fd-482f-b003-a9e21241a182 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.041791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.042455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.042738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.043014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] PATH_INFO: `/v3/users/fd3faae32a6046c5acf1bbad810329f6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.044158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] RBAC: Authorizing `identity:get_user(user_id=fd3faae32a6046c5acf1bbad810329f6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.045670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.047565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 126/263] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/users/fd3faae32a6046c5acf1bbad810329f6 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:14.053037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:14.096803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:14.117652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a947d8-c5f4-4b0b-9e79-bb5cedf9280c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.166881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fcee5-f3bf-4577-8e50-514f7ca9be75 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.202607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.203221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.203503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.203780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.251253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.251879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.252165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.252443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.277455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.279032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.320740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:14.321784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:14.404751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 127/264] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:47:14.411855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:14.438542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.440019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.596676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 138/265] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:14.602253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:14.610865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181cbc1-1a2b-44bb-81b8-a0156651cb13 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.660576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.661244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.661525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.661800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.662973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.663955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:14.666314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.668129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 128/266] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:14 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:14.674570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:14.686338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:14.704144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6e09-d076-4808-8529-7dda3b1cca7e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.739767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.740532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.740906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.741275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.742613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.744207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.750338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 139/267] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:14 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:14.752305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:14.783017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d52769-a552-45f1-9592-5eeaf52b0780 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.820076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.820689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.820974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.821252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.821988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] RBAC: Adding query filter params (role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=fd3faae32a6046c5acf1bbad810329f6) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:14.822857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.824440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd71fa-9b2f-43ee-ba67-a32fdda5619e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.834107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 129/268] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:14 2022] GET /identity/v3/role_assignments?user.id=fd3faae32a6046c5acf1bbad810329f6&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=399f7b70c70b4f09addcf156bf604eab => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:14.884020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19994b4b-c1ba-46a3-ad1e-1b27113f7aec None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:14.932362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:14.937491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:14.938150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:14.942403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:14.942686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:14.943516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=fd3faae32a6046c5acf1bbad810329f6) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:14.943827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:14.945429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83612f90-937b-410f-994b-4753aae17e34 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:14.952529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 140/269] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:14 2022] GET /identity/v3/role_assignments?user.id=fd3faae32a6046c5acf1bbad810329f6&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:15.362921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479061fc-f3d4-401f-9757-276436568b08 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.363422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479061fc-f3d4-401f-9757-276436568b08 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.363496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479061fc-f3d4-401f-9757-276436568b08 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.364951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 130/270] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:15.370923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f51ca-cae8-453a-8449-18fa99af52ca None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.371213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f51ca-cae8-453a-8449-18fa99af52ca None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.371495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f51ca-cae8-453a-8449-18fa99af52ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.393064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-286f51ca-cae8-453a-8449-18fa99af52ca None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:15.394283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-286f51ca-cae8-453a-8449-18fa99af52ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:15.484270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 141/271] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:15.589962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c9152-c12c-4254-a54f-6012b5e63a87 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.590296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c9152-c12c-4254-a54f-6012b5e63a87 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.590520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c9152-c12c-4254-a54f-6012b5e63a87 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.592104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 131/272] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:15.597457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11acc1-d015-429c-877d-16b621df8b44 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.597742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11acc1-d015-429c-877d-16b621df8b44 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.598040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11acc1-d015-429c-877d-16b621df8b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.619550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7d11acc1-d015-429c-877d-16b621df8b44 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:15.620672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7d11acc1-d015-429c-877d-16b621df8b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:15.701483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 142/273] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:15.791556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c9152-c12c-4254-a54f-6012b5e63a87 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:15.877132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:15.910416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c9152-c12c-4254-a54f-6012b5e63a87 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:15.945860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:15.946297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe04094-b71a-43d6-b450-05f9782b8f77 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.946735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.946808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.948957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.949080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe04094-b71a-43d6-b450-05f9782b8f77 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.949080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe04094-b71a-43d6-b450-05f9782b8f77 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.949519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 143/274] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:15.954553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:15.955067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:15.955543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:15.957663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:15.963422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:15.964296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:15.965113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 132/275] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:15.970339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:15.978252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:15.980436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:16.034987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.058162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 144/276] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:16.059171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.089111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cdb57-5960-4d2e-9f3c-805cf64894ba None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.125266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.131712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.132340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.132684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.133021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.133892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:16.134413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.136212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.145450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 133/277] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:16.155079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.177197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536b1fd-d1a5-43c3-8680-451993da41f3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.211985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836b885-f5b2-422e-8394-46d69757c9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.212612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836b885-f5b2-422e-8394-46d69757c9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.213434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836b885-f5b2-422e-8394-46d69757c9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.213721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836b885-f5b2-422e-8394-46d69757c9a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.223616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836b885-f5b2-422e-8394-46d69757c9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-d836b885-f5b2-422e-8394-46d69757c9a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:16.228462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:16.229556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:16.230573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:16.234408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 145/278] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:16.236029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630b32-e069-424a-9f3d-6384659fbc19 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.236326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630b32-e069-424a-9f3d-6384659fbc19 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.236604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630b32-e069-424a-9f3d-6384659fbc19 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.237870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 146/279] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:16.239115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82630b32-e069-424a-9f3d-6384659fbc19 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.275575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d60ec5-9d36-4486-a2b2-de841c13f571 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.290710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.311004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45a8bc-e8d3-476d-b0dc-bd2127114e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.311863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45a8bc-e8d3-476d-b0dc-bd2127114e3c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.312288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45a8bc-e8d3-476d-b0dc-bd2127114e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.312707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45a8bc-e8d3-476d-b0dc-bd2127114e3c None admin] PATH_INFO: `/v3/users/c3697c1cbbf5479ba36b55de55d59582` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.327696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45a8bc-e8d3-476d-b0dc-bd2127114e3c None admin] RBAC: Authorizing `identity:get_user(user_id=c3697c1cbbf5479ba36b55de55d59582)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.329483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45a8bc-e8d3-476d-b0dc-bd2127114e3c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.331538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 134/280] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/users/c3697c1cbbf5479ba36b55de55d59582 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:16.336461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.336709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.336986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.347436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82630b32-e069-424a-9f3d-6384659fbc19 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.354814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:16.356004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:16.381308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.381981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.382214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.382484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.383133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:16.383429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.384855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.392584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 147/281] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:16.393345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.402692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.423619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 135/282] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:16.424448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.496086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24ae7f-6ba0-4101-8402-97c4d870f7a2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.503201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.532213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.532842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.533120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.533389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.534840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.536620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.537077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90af4f6-d76b-4282-88ef-17e06c38a1ea None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.538377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 148/283] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:16.538977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.576746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.577666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.578172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.578696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.579481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, user.id=c3697c1cbbf5479ba36b55de55d59582, scope.system=all) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:16.579792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.581198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.590175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 136/284] 10.222.0.50 () {56 vars in 1308 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/role_assignments?user.id=c3697c1cbbf5479ba36b55de55d59582&scope.system=all&role.id=6768099ea599447f9de57814f623441c => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:16.591054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.660230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19b006-c581-4b86-ba71-bb6096d660f5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.671171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.700845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.701495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.701771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.702068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] PATH_INFO: `/v3/users/2678daf5b93c47d48cd48802e3484e00` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.703236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] RBAC: Authorizing `identity:get_user(user_id=2678daf5b93c47d48cd48802e3484e00)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.704712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.706685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 149/285] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/users/2678daf5b93c47d48cd48802e3484e00 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:16.707885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.739801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058eabd-9151-4791-a65a-2dbb4acc23d7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.756243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.786682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.787651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.788124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.788595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.789458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:16.791266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:16.819000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 137/286] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:16 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:16.820750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d215c0-c97e-444a-b9f9-84541b01728e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:16.827130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.864016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:16.864647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:16.864932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:16.865206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:16.878942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:16.885051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:16.886056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:16.886847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 150/287] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:16.892368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:16.977913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:16.983212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cbd24-74f7-4257-b282-28c8040b9729 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.019920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b35bd-9b7b-4d2e-8ab3-860414940c01 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.022023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:17.032523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.033466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.034127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.034407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.065436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.066103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.066383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.066674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.067463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:17.067761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.069163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.087771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 151/288] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:17.094347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:17.124423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.126254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.224995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009c418-cd26-4f03-b136-923ac9f82e4c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.250041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:17.260547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 138/289] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:17.261913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:17.266366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.267009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.267280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.267545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] PATH_INFO: `/v3/users/ed5b761801074363943fd9f724baa54f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.268677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] RBAC: Authorizing `identity:get_user(user_id=ed5b761801074363943fd9f724baa54f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.270173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.272043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 152/290] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:17 2022] GET /identity/v3/users/ed5b761801074363943fd9f724baa54f => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:17.278311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:17.320486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:17.383220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0705862-0e69-479e-8b41-203a1bc05b00 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.403310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce09fc4-71a1-4a60-8c23-f642d93b15cb None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.434721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.435975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.436614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.438066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.440123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.443005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.445714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 139/291] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:17 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:17.452426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:17.461346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.462418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.463510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.463998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.465400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.467104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.468955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 153/292] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:17 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:17.475848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:17.579522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:17.644379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9abcb-f718-4878-a44b-e59b650a83fb None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.667304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:17.672018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b582034-3f6b-443c-967b-eb414188c0ff None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:17.712052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.712667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.713462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.713732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.714487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=2678daf5b93c47d48cd48802e3484e00) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:17.714787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.716212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fea6f-abff-45c3-844d-cad22a085eb0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.720639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028d845-298a-4ce4-85d7-c3cd81610dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:17.721525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028d845-298a-4ce4-85d7-c3cd81610dcc None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:17.721977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028d845-298a-4ce4-85d7-c3cd81610dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:17.722445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028d845-298a-4ce4-85d7-c3cd81610dcc None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/ed5b761801074363943fd9f724baa54f/roles/e99d1aaf2c754f72a1b1f8c6bba548e1` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:17.724675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028d845-298a-4ce4-85d7-c3cd81610dcc None admin] RBAC: Authorizing `identity:create_grant(project_id=a1259daa670c46fcacf1d9b1449f2537, user_id=ed5b761801074363943fd9f724baa54f, role_id=e99d1aaf2c754f72a1b1f8c6bba548e1)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:17.726329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028d845-298a-4ce4-85d7-c3cd81610dcc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:17.731719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 140/293] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:17 2022] GET /identity/v3/role_assignments?user.id=2678daf5b93c47d48cd48802e3484e00&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=6768099ea599447f9de57814f623441c => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:17.744704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 154/294] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:17 2022] PUT /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537/users/ed5b761801074363943fd9f724baa54f/roles/e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:18.784347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec81f6e-4119-4ee8-a774-eed738509601 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:18.784846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec81f6e-4119-4ee8-a774-eed738509601 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:18.784953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec81f6e-4119-4ee8-a774-eed738509601 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:18.786452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 141/295] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:18.792149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:18.792536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:18.792703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:18.811470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:18.812717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:18.886020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 155/296] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:19.146031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec81f6e-4119-4ee8-a774-eed738509601 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:19.255160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec81f6e-4119-4ee8-a774-eed738509601 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:19.290361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:19.296219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef5df4-f274-4ad4-99e4-be8375cb3681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:19.296859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef5df4-f274-4ad4-99e4-be8375cb3681 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.297136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef5df4-f274-4ad4-99e4-be8375cb3681 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.297408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef5df4-f274-4ad4-99e4-be8375cb3681 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.309515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef5df4-f274-4ad4-99e4-be8375cb3681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-ffef5df4-f274-4ad4-99e4-be8375cb3681 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:19.316163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:19.317029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:19.317837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 142/297] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:19.324193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:19.430333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:19.440210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8061c2f-2dae-49aa-b1ba-409c982153cc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:19.480656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:19.481272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.481551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.481826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.482521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] RBAC: Adding query filter params (name=member) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:19.482818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:19.484184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:19.488199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe38dc-9f09-4b54-a53f-b14f7620221d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.488713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe38dc-9f09-4b54-a53f-b14f7620221d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.489211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe38dc-9f09-4b54-a53f-b14f7620221d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.492137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 143/298] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:19.492894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 156/299] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:19.498507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01273b-22b2-4c8d-8dcf-6079f5e6737b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.498647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:19.498857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01273b-22b2-4c8d-8dcf-6079f5e6737b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.499436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01273b-22b2-4c8d-8dcf-6079f5e6737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.519789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3b01273b-22b2-4c8d-8dcf-6079f5e6737b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:19.520952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3b01273b-22b2-4c8d-8dcf-6079f5e6737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:19.591495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 144/300] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:19.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c510-d6af-4ab1-b9d9-752af9e2fd7f None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c510-d6af-4ab1-b9d9-752af9e2fd7f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c510-d6af-4ab1-b9d9-752af9e2fd7f None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 145/301] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:19.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.601210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.617694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:19.618889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:19.621091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a425c29-71f0-4d0b-bc6d-0d88fc153643 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:19.662558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:19.663431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.664004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.664483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] PATH_INFO: `/v3/users/c3697c1cbbf5479ba36b55de55d59582` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.665850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] RBAC: Authorizing `identity:get_user(user_id=c3697c1cbbf5479ba36b55de55d59582)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:19.667643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:19.669665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 157/302] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:19 2022] GET /identity/v3/users/c3697c1cbbf5479ba36b55de55d59582 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:19.677408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:19.704849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 146/303] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:19.706237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:19.792294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fc1dd-04a8-49a6-b8ec-da414b6d9b77 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:19.827107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:19.827757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:19.828035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:19.828308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] PATH_INFO: `/v3/system/users/c3697c1cbbf5479ba36b55de55d59582/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:19.829707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c3697c1cbbf5479ba36b55de55d59582, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:19.831236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:19.848863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 158/304] 10.222.0.50 () {58 vars in 1334 bytes} [Sat Sep 24 04:47:19 2022] PUT /identity/v3/system/users/c3697c1cbbf5479ba36b55de55d59582/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 04:47:19.895689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:19.955966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:19.986186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.011557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d94ce2-6f0e-488b-811f-25106498f54b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.051823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.052472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.052754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.053030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.066984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:20.072929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:20.073862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:20.074660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:20.076156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 147/305] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:20.080867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.106385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:20.119909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971444e-4ba6-42ce-912e-a92c1936aecc None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.158647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e68541-bb1e-425f-93fd-5cfa18d6f345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.159281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68541-bb1e-425f-93fd-5cfa18d6f345 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.159557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68541-bb1e-425f-93fd-5cfa18d6f345 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.159825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e68541-bb1e-425f-93fd-5cfa18d6f345 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.160938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68541-bb1e-425f-93fd-5cfa18d6f345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:20.162794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e68541-bb1e-425f-93fd-5cfa18d6f345 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:20.164720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 159/306] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:20.166782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffba5-43de-4aee-93d1-0cce1c53cdd0 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.167061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffba5-43de-4aee-93d1-0cce1c53cdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.167326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ffba5-43de-4aee-93d1-0cce1c53cdd0 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.168673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 160/307] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:20.170616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffba5-43de-4aee-93d1-0cce1c53cdd0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.193763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24bfbe-dbda-4dce-8ce9-a0c3ab8eee49 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.207363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:20.229699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.230340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.230614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.230881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.231976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:20.232280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:20.234121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5c4dd-ff7b-48ac-927d-bea0ba45b5c6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:20.243153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 148/308] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:20.244492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.244779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.245046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.264011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:20.265183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:20.283085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ffba5-43de-4aee-93d1-0cce1c53cdd0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.318264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.318871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.319205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.319467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] PATH_INFO: `/v3/groups` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.320103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:20.321561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:20.346029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 161/309] 10.222.0.50 () {60 vars in 1134 bytes} [Sat Sep 24 04:47:20 2022] POST /identity/v3/groups => generated 229 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:47:20.346029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.346283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 149/310] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:20.366435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:20.467466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db29fcd-cb5b-40bd-8bf3-9d65b395862a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.506719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.507374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.507654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.507929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] PATH_INFO: `/v3/users/716da45c2ad64ebd87d0009154c1acec` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.523038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] RBAC: Authorizing `identity:get_user(user_id=716da45c2ad64ebd87d0009154c1acec)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:20.524518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:20.526774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 162/311] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:20 2022] GET /identity/v3/users/716da45c2ad64ebd87d0009154c1acec => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:20.532786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.608214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.640228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d47c1-230d-4331-b79c-ddfb9c73f54e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.648370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:20.686216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.686828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.687107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.687379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.688112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] RBAC: Adding query filter params (role.id=e99d1aaf2c754f72a1b1f8c6bba548e1, user.id=716da45c2ad64ebd87d0009154c1acec, scope.system=all) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:20.688418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:20.689869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:20.700140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 150/312] 10.222.0.50 () {56 vars in 1308 bytes} [Sat Sep 24 04:47:20 2022] GET /identity/v3/role_assignments?user.id=716da45c2ad64ebd87d0009154c1acec&scope.system=all&role.id=e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:20.726726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:20.746884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f5b2f-d736-4ef1-897b-f6e3d115cbe0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.788638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.789284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.789628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.789908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.802419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:20.807779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:20.808729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:20.809496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 163/313] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:20.815519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:20.928506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ffdaa-bbf1-4cbb-be19-1c179022b176 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:20.963540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:20.964480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:20.964948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:20.965430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:20.966331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:20.966818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:20.968363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:20.976835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 151/314] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:20.983298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:21.092277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e210a-efdf-402d-a38e-ac980fd8cd00 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:21.122531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:21.128939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:21.129817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:21.130325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:21.130810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] PATH_INFO: `/v3/users/ed5b761801074363943fd9f724baa54f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:21.132121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] RBAC: Authorizing `identity:get_user(user_id=ed5b761801074363943fd9f724baa54f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:21.133762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:21.135980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 164/315] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:20 2022] GET /identity/v3/users/ed5b761801074363943fd9f724baa54f => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:21.143051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:21.176109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:21.251406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36a0e3-2633-404f-a629-f35c4c3688b1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:21.286855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:21.287833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:21.288312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:21.288804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:21.362778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:21.364625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:21.402403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:21.476708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 152/316] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:21.482981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:21.587578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f5162-5d36-44dd-bd23-1a1a1a405e31 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:21.622468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06f6c4-b057-4865-8154-63da1adfd149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:21.623511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06f6c4-b057-4865-8154-63da1adfd149 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:21.624036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06f6c4-b057-4865-8154-63da1adfd149 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:21.624769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06f6c4-b057-4865-8154-63da1adfd149 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:21.626212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06f6c4-b057-4865-8154-63da1adfd149 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:21.627842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06f6c4-b057-4865-8154-63da1adfd149 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:21.633391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 165/317] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:21 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:21.636254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:21.747391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682349a-896b-4170-a206-9c4e15a2ecac None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:21.778831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:21.784716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:21.785724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:21.786207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:21.786669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:21.787410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] RBAC: Adding query filter params (role.id=e99d1aaf2c754f72a1b1f8c6bba548e1, scope.project.id=a1259daa670c46fcacf1d9b1449f2537, user.id=ed5b761801074363943fd9f724baa54f) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:21.787718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:21.789153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a9ced-2452-4e9f-8a30-fbc052dab023 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:21.802171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 153/318] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:21 2022] GET /identity/v3/role_assignments?user.id=ed5b761801074363943fd9f724baa54f&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:22.101499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8c1a-d494-438d-9d65-83e2b9ae990f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.102032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8c1a-d494-438d-9d65-83e2b9ae990f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.102112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8c1a-d494-438d-9d65-83e2b9ae990f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.103583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 166/319] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:22.109088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.109563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.109638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.127895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:22.129067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:22.195609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 154/320] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:22.459375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8c1a-d494-438d-9d65-83e2b9ae990f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:22.525413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:22.561976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8c1a-d494-438d-9d65-83e2b9ae990f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:22.597025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab436e5-eed5-42d5-a7f1-231c88e51016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:22.597682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab436e5-eed5-42d5-a7f1-231c88e51016 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.598047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab436e5-eed5-42d5-a7f1-231c88e51016 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.598324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab436e5-eed5-42d5-a7f1-231c88e51016 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.610567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab436e5-eed5-42d5-a7f1-231c88e51016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-aab436e5-eed5-42d5-a7f1-231c88e51016 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:22.615629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:22.616552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:22.617335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:22.618741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 167/321] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:22.623624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:22.714317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5b6ef-4e22-4126-85b7-9b1166e67e77 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.714529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5b6ef-4e22-4126-85b7-9b1166e67e77 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.714783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5b6ef-4e22-4126-85b7-9b1166e67e77 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.716122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 168/322] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:22.720875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.721157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.721427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.732733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24857d-3429-496d-893b-0cbb5892ae9b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:22.739490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:22.740686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:22.758685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:22.771034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:22.771636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.771911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.772178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.772837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:22.773138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:22.774526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:22.783244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 155/323] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:22.788896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:22.812396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 169/324] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:22.895473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442e921-a0bf-4a7d-936f-75cf0768508b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:22.928993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22582dbe-99b5-45de-b537-af9aadcb510b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:22.929929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22582dbe-99b5-45de-b537-af9aadcb510b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.930450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22582dbe-99b5-45de-b537-af9aadcb510b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.930890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22582dbe-99b5-45de-b537-af9aadcb510b None admin] PATH_INFO: `/v3/users/c3697c1cbbf5479ba36b55de55d59582` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.932177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22582dbe-99b5-45de-b537-af9aadcb510b None admin] RBAC: Authorizing `identity:get_user(user_id=c3697c1cbbf5479ba36b55de55d59582)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:22.933847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22582dbe-99b5-45de-b537-af9aadcb510b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:22.940252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 156/325] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:22 2022] GET /identity/v3/users/c3697c1cbbf5479ba36b55de55d59582 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:22.941674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:22.952868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d9a8-e596-4288-9921-b1603e4afe88 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.953337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d9a8-e596-4288-9921-b1603e4afe88 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.953613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d9a8-e596-4288-9921-b1603e4afe88 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.955074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 157/326] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:22.959720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:22.959997 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:22.960262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:22.979144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:22.980200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:23.008493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:23.032609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:23.051064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 158/327] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:23.063067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8ec99-7923-45c6-8f2f-fd0f2e6440a1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.082531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:23.097182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.097794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.098103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.098379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.099097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] RBAC: Adding query filter params (role.id=6768099ea599447f9de57814f623441c, user.id=c3697c1cbbf5479ba36b55de55d59582, scope.system=all) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:23.099398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:23.100852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:23.113133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 170/328] 10.222.0.50 () {56 vars in 1308 bytes} [Sat Sep 24 04:47:22 2022] GET /identity/v3/role_assignments?user.id=c3697c1cbbf5479ba36b55de55d59582&scope.system=all&role.id=6768099ea599447f9de57814f623441c => generated 510 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:23.195565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b652a-d9dc-433d-b4a7-133e71e9576c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.233905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.234843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.235265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.235681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.237049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:23.238791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:23.240994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 159/329] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:23.247167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:23.328221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:23.360973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0180669-325d-414e-92d5-9ccdb8dcb0f6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.376182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:23.390408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:23.397137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.397739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.398034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.398318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] PATH_INFO: `/v3/groups` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.398916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:23.400336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:23.414357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 171/330] 10.222.0.50 () {60 vars in 1134 bytes} [Sat Sep 24 04:47:23 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:47:23.443557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0c9c9-30bb-4a5e-a11a-582033f52d70 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.481527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.482202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.482482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.482756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.494854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:23.500010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:23.500915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:23.501699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 160/331] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:23.507461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:23.613804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369fdb-ac03-42f2-aad4-9b4fec85004e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.652158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.652778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.653053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.653325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.653988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:23.654327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:23.655693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:23.667244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 172/332] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:23.672794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:23.779875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184b159-270d-4830-9dff-e3e208aedaf3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.794126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:23.816156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bf11b-0618-456f-9f64-8cf04b4abffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.817045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf11b-0618-456f-9f64-8cf04b4abffa None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.817470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf11b-0618-456f-9f64-8cf04b4abffa None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.817739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf11b-0618-456f-9f64-8cf04b4abffa None admin] PATH_INFO: `/v3/users/716da45c2ad64ebd87d0009154c1acec` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.818909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bf11b-0618-456f-9f64-8cf04b4abffa None admin] RBAC: Authorizing `identity:get_user(user_id=716da45c2ad64ebd87d0009154c1acec)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:23.820342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bf11b-0618-456f-9f64-8cf04b4abffa None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:23.822772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 161/333] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:23 2022] GET /identity/v3/users/716da45c2ad64ebd87d0009154c1acec => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:23.828036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:23.865046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:23.938912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f00be-a83b-41d0-8dfb-2a43fbfbf8cc None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:23.974379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21df9a-3048-4622-ae0a-a0a6c308b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:23.975044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21df9a-3048-4622-ae0a-a0a6c308b291 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:23.975329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21df9a-3048-4622-ae0a-a0a6c308b291 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:23.975604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21df9a-3048-4622-ae0a-a0a6c308b291 None admin] PATH_INFO: `/v3/system/users/716da45c2ad64ebd87d0009154c1acec/roles/e99d1aaf2c754f72a1b1f8c6bba548e1` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:23.977074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21df9a-3048-4622-ae0a-a0a6c308b291 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=716da45c2ad64ebd87d0009154c1acec, role_id=e99d1aaf2c754f72a1b1f8c6bba548e1)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:23.979029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e21df9a-3048-4622-ae0a-a0a6c308b291 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:23.988210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 173/334] 10.222.0.50 () {58 vars in 1334 bytes} [Sat Sep 24 04:47:23 2022] PUT /identity/v3/system/users/716da45c2ad64ebd87d0009154c1acec/roles/e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:26.373386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402105ab-26d0-4a46-a48b-e43c10b9c998 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:26.374686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402105ab-26d0-4a46-a48b-e43c10b9c998 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:26.375166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402105ab-26d0-4a46-a48b-e43c10b9c998 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:26.376992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 162/335] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:26.382563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cfbc-6ff6-4ad7-9b42-4cb9a9e55226 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:26.383425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cfbc-6ff6-4ad7-9b42-4cb9a9e55226 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:26.383909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540cfbc-6ff6-4ad7-9b42-4cb9a9e55226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:26.405574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5540cfbc-6ff6-4ad7-9b42-4cb9a9e55226 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:26.407665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5540cfbc-6ff6-4ad7-9b42-4cb9a9e55226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:26.480484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 174/336] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:26.517899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09535bb2-a162-416d-a32b-269452d9224c None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:26.518547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09535bb2-a162-416d-a32b-269452d9224c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:26.519053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09535bb2-a162-416d-a32b-269452d9224c None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:26.520903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 163/337] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:26.526719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:26.527248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:26.527753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:26.549927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:26.551552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:26.582192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:26.637702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 175/338] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:26.994252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09535bb2-a162-416d-a32b-269452d9224c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.131008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09535bb2-a162-416d-a32b-269452d9224c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:27.168810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:27.169450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.169727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.170012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.183719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:27.189260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:27.190237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:27.191003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 164/339] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:27.196267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.206428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.273822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:27.345524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196cba9-473d-4bd9-bce8-185c1f3265a1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:27.347525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b69d6-d5a7-4bc0-afc8-354910dab8bd None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:27.392854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:27.393458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:27.393458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.393894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.393963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.394057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.394126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.394206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.394747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:27.395411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:27.396496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdddb9-9e3f-46e8-a16c-065ce38da237 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:27.405566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 176/340] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:27.408961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6074bb-60cc-407c-b2d9-191cc4c6fc8f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.408961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6074bb-60cc-407c-b2d9-191cc4c6fc8f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.408961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6074bb-60cc-407c-b2d9-191cc4c6fc8f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.408961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 177/341] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:27.410257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:27.413836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ff838-714f-4bbf-8bb7-000f471f8052 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.415648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ff838-714f-4bbf-8bb7-000f471f8052 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.415648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ff838-714f-4bbf-8bb7-000f471f8052 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.415823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 178/342] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:27.417466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7fbcb-44e8-47ed-b078-9a8f15b22f4d None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.417751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7fbcb-44e8-47ed-b078-9a8f15b22f4d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.418042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7fbcb-44e8-47ed-b078-9a8f15b22f4d None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.419427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 179/343] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:27.419776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:27.420615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:27.421380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:27.421969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba8f5f-de6c-4a3d-b8e4-4662146c2618 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.421969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba8f5f-de6c-4a3d-b8e4-4662146c2618 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.421969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba8f5f-de6c-4a3d-b8e4-4662146c2618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.424177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 165/344] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:27.425156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.442680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-10ba8f5f-de6c-4a3d-b8e4-4662146c2618 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:27.443865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-10ba8f5f-de6c-4a3d-b8e4-4662146c2618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:27.466069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:27.526670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 180/345] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:27.527827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e9a53-bc87-4620-a8df-31ff4e7a9423 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.528109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e9a53-bc87-4620-a8df-31ff4e7a9423 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.528376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e9a53-bc87-4620-a8df-31ff4e7a9423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.539550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:27.544303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770c3d5d-b758-4f0b-be19-300c82587489 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:27.551510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ef8e9a53-bc87-4620-a8df-31ff4e7a9423 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:27.552704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ef8e9a53-bc87-4620-a8df-31ff4e7a9423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:27.601945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:27.602581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.602852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.603116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.646912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 181/346] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:27.648409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.648696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.648967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.672182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:27.673786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:27.682300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:27.683866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:27.761327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 182/347] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:27.762297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.783918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:27.793019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:27.797602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 166/348] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:27.804315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.895509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3641b59-5b9c-4ef0-b34c-c71e63e480b7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:27.918275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f138383-6444-4dc0-936f-f5863637635c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:27.941558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:27.942547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.942834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.943140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.944458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] RBAC: Adding query filter params (name=member) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:27.944772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:27.948341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4a255-7bce-479c-9889-45b97f9a5869 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:27.958305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e843dac-5336-4c9c-ac6f-cba20e89800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:27.958937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e843dac-5336-4c9c-ac6f-cba20e89800c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.959208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e843dac-5336-4c9c-ac6f-cba20e89800c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.959507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 183/349] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:27.959750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e843dac-5336-4c9c-ac6f-cba20e89800c None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.960897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e843dac-5336-4c9c-ac6f-cba20e89800c None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:27.961002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0ba4-b18c-43d0-9024-55645d0f0571 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.961170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0ba4-b18c-43d0-9024-55645d0f0571 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.961451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed0ba4-b18c-43d0-9024-55645d0f0571 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.962499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e843dac-5336-4c9c-ac6f-cba20e89800c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:27.963077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 184/350] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:27.964378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 167/351] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:27 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:27.966838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0ba4-b18c-43d0-9024-55645d0f0571 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:27.970788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:27.971062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:27.971326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:27.999728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:28.001682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:28.073963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.081572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 168/352] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:28.082454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.084264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.123066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed0ba4-b18c-43d0-9024-55645d0f0571 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.179052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.179467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.179778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.180006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.252583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec7703-6045-4e8f-aa04-112f03f22e53 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.266307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:28.268286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:28.291937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.292569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.292844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.293109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.299210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:28.301100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:28.303021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 169/353] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:28.303916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.355913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.385273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 185/354] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:28.386734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.389452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.435339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07021ea5-7af3-4bcb-b03c-eba8507c3cbc None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.471816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.472439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.472710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.472976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.485084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:28.490468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:28.491542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 170/355] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:28.492458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.521064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93719d8-5ac5-474b-b373-2f2416281853 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.567536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.568173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.568449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.568721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.581726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:28.587239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:28.588237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:28.589013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 186/356] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:28.589657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.615990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06868e0e-82ae-40dc-98cd-235931c4fdaf None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.625391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.644086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.665152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.665772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.667114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.667114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.683177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:28.691158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:28.693012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 171/357] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:28.693937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.711565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d3d57-cb85-4892-b935-e2eb0d65f3e2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.755158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.755776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.756055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.756324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.757100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=595b8e1093aa49399a51e159588da573) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:28.757345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:28.758890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:28.769472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 187/358] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=399f7b70c70b4f09addcf156bf604eab => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:28.770410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.815388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87c719-559d-4c9a-950c-6737a5bc3bc5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.854808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.855880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.856310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.856588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.875217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:28.882249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:28.883211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:28.883983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:28.884585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 172/359] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:28.885563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:28.903630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.912356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281cea72-942f-4739-ad09-5f3b5e85281d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:28.947916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:28.964575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:28.965214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:28.965490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:28.965756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:28.967062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:28.968936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:28.970788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 188/360] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:28.971359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.024746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1b265-7816-46ee-be61-c2a824dbb836 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.076361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.076976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.077252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.077519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.078925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:29.079423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.080612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.093341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 173/361] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:29.094200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.099996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75938-2a65-4884-a9dc-6167c4f3e8bf None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.144930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.145593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.145875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.146180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.146845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:29.147146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.149222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.157675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 189/362] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:29.158677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.192571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:29.222581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f36a-822e-47b5-9e24-11e28d7c604b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.225876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:29.272089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.272866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.273255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.273620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.274413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:29.274813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.276395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.285348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a5f15-c93f-4c0b-990c-da69e805f1db None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.289158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 174/363] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:29.290429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.328703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.329322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.329599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.329982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.330585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:29.330880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.332323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.341583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 190/364] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:29.342485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.413326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cff8b-e126-4d88-bb49-b56c88d9223f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.454554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.455189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.455189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.455356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.456470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.458057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.459631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 175/365] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:29.460380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.467673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:29.481964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec80165-fb3a-4a79-bd36-7e43c2af68b5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.496972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:29.533554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.534263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.534561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.534982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] PATH_INFO: `/v3/users/716da45c2ad64ebd87d0009154c1acec` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.536430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] RBAC: Authorizing `identity:get_user(user_id=716da45c2ad64ebd87d0009154c1acec)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.538232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.540027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 191/366] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/users/716da45c2ad64ebd87d0009154c1acec => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:29.540740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.583532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3e55c-4d34-4cdb-a3f3-e12aaf37f8a7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.629251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.630038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.630178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.630415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.676041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7b79-1f70-4234-a969-6461c0d75289 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.723756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.724746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.725225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.725562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.727766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.730216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.763597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:29.774667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:29.816263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:29.817765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:29.843663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 176/367] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:29.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.939573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 192/368] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:29.940589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:29.951718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c04c9-86db-4cb3-b9ef-570bc5b65c06 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:29.986457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:29.987065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:29.987340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:29.987609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:29.990509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.004827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.052730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460d587-13c8-48ec-9a7d-ed724490f74f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.061699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.064483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.089865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.090497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.090776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.091050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.091770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=6768099ea599447f9de57814f623441c, scope.project.id=595b8e1093aa49399a51e159588da573) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:30.092072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.093505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.099974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 193/369] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=6768099ea599447f9de57814f623441c => generated 260 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.100924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.190303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 177/370] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:30.191198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.231113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.246086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238213eb-9b6e-45ad-9a1f-cb01e0a7e182 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.272991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.288357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.289662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.290190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.290740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.291875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] RBAC: Adding query filter params (role.id=e99d1aaf2c754f72a1b1f8c6bba548e1, user.id=716da45c2ad64ebd87d0009154c1acec, scope.system=all) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:30.292409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.294164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.308807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 194/371] 10.222.0.50 () {56 vars in 1308 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/role_assignments?user.id=716da45c2ad64ebd87d0009154c1acec&scope.system=all&role.id=e99d1aaf2c754f72a1b1f8c6bba548e1 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.309375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba040fa-ca67-488a-b203-cae21f2bb00b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.309647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.359653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.361174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.361629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.361913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.363140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.364620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.366963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 178/372] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.367809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.484662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9bf80-b198-4c87-8516-9d42dcd3724e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.535617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09787236-6b15-42ef-948e-fa2b31b71f00 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.537175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.540125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.540906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.541806 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.542200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.543506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.545733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.547615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 195/373] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.548548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.580843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.581482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.581761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.582070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] PATH_INFO: `/v3/projects/f9c9e5a50c664c33aa419611e20c9c00` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.583208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c9e5a50c664c33aa419611e20c9c00)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.584911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.586873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 179/374] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/projects/f9c9e5a50c664c33aa419611e20c9c00 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.587750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.607268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.674228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773217c-26cd-4fe2-84ec-08af424441b9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.708192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e36fee-2d52-4e8c-8dac-2b538cc3f2a4 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.715526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.716570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.717039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.717512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.718963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.720719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.724737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 196/375] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.725803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.761423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.761423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.762264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.763022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.764581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.766802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.769636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 180/376] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.770804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.820319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.864341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ddf28-2e83-4266-ab9c-c4cd6385ca8e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.886496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:30.889099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709abe1-189c-456e-b322-00ebe785a3d5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:30.907428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.908066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.908342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.908613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] PATH_INFO: `/v3/groups/e01927bc54e044cba76b0a9d54171881` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.917626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] RBAC: Authorizing `identity:get_group(group_id=e01927bc54e044cba76b0a9d54171881)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.919319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.921235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 197/377] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/groups/e01927bc54e044cba76b0a9d54171881 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.921918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:30.932378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:30.933341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:30.933831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:30.934439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:30.935383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=a1259daa670c46fcacf1d9b1449f2537) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:30.935887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:30.937518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:30.943624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 181/378] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=399f7b70c70b4f09addcf156bf604eab => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:30.944637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:31.109320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999183a-faff-4cb2-94bc-30e24423b29c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:31.148210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c8d336-04ad-4ccb-a308-911912f3665b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:31.150206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:31.150854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:31.151628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:31.151970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:31.152283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:31.153097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=6768099ea599447f9de57814f623441c, scope.project.id=a1259daa670c46fcacf1d9b1449f2537) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:31.153403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:31.154927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5e227-b8a3-4416-9c72-6116ac8231ba None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:31.161392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 198/379] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=6768099ea599447f9de57814f623441c => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:31.162678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a549-57ab-40e4-9733-8f578732bbcd None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:31.162962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a549-57ab-40e4-9733-8f578732bbcd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:31.163232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659a549-57ab-40e4-9733-8f578732bbcd None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:31.164690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 199/380] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:31.170324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:31.170693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:31.171036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:31.193673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:31.193908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:31.194375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:31.194647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:31.194965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:31.195475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:31.195752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] RBAC: Adding query filter params (group.id=e01927bc54e044cba76b0a9d54171881, role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=f9c9e5a50c664c33aa419611e20c9c00) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:31.196053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:31.197821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:31.204614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 182/381] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:30 2022] GET /identity/v3/role_assignments?group.id=e01927bc54e044cba76b0a9d54171881&scope.project.id=f9c9e5a50c664c33aa419611e20c9c00&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:31.248559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:31.283477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 200/382] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:31.633931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:31.764921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7db92-5ae6-40ac-8029-5ddb83f08b78 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:31.812032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:31.812664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:31.812937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:31.813204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:31.827702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:31.832994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:31.833867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:31.834712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 183/383] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:31.840726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:32.003050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbbb54-817a-49ee-b577-fc9a9babd720 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:32.038826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:32.039361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:32.039638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:32.039909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:32.040564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:32.040861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:32.042772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:32.051294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 201/384] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:32.056485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:32.116000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:32.219307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62b4389-3bc2-4ed2-b3f2-57ea9889da79 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:32.254339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:32.254952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:32.255232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:32.255504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:32.256616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:32.258081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:32.260495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 184/385] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:32 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:32.267097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:32.312698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:32.382897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79e070-84d2-41a2-91d9-4a8368b772b5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:32.419594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82957a6c-45b1-4c44-834d-31ae7cc16a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:32.420227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82957a6c-45b1-4c44-834d-31ae7cc16a27 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:32.420505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82957a6c-45b1-4c44-834d-31ae7cc16a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:32.420779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82957a6c-45b1-4c44-834d-31ae7cc16a27 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:32.421977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82957a6c-45b1-4c44-834d-31ae7cc16a27 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:32.423659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82957a6c-45b1-4c44-834d-31ae7cc16a27 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:32.425333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 202/386] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:32 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:32.430970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:32.540243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:32.548251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecdf5a-22bf-4a8f-b787-92f545fe9e3a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:32.596559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b1cb1-371c-41dd-aa63-dca3df14d4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:32.597222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1cb1-371c-41dd-aa63-dca3df14d4e6 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:32.597499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1cb1-371c-41dd-aa63-dca3df14d4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:32.597772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1cb1-371c-41dd-aa63-dca3df14d4e6 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573/groups/dc46ba8ff0d34d51bee067470f850611/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:32.599984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b1cb1-371c-41dd-aa63-dca3df14d4e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=595b8e1093aa49399a51e159588da573, group_id=dc46ba8ff0d34d51bee067470f850611, role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:32.601419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b1cb1-371c-41dd-aa63-dca3df14d4e6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:32.622557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 185/387] 10.222.0.50 () {58 vars in 1442 bytes} [Sat Sep 24 04:47:32 2022] PUT /identity/v3/projects/595b8e1093aa49399a51e159588da573/groups/dc46ba8ff0d34d51bee067470f850611/roles/399f7b70c70b4f09addcf156bf604eab => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:32.949217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22743a81-d260-4f0b-95cf-e5b82f1e0167 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:32.949898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22743a81-d260-4f0b-95cf-e5b82f1e0167 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:32.949898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22743a81-d260-4f0b-95cf-e5b82f1e0167 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:32.951267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 203/388] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:32.956544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:32.957048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:32.957216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:32.979653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:32.980846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:33.064944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 186/389] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:33.332738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22743a81-d260-4f0b-95cf-e5b82f1e0167 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:33.470367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22743a81-d260-4f0b-95cf-e5b82f1e0167 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:33.472912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:33.516577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0144952-b95f-4087-88d4-a36b2d308c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:33.517223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144952-b95f-4087-88d4-a36b2d308c63 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:33.517503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144952-b95f-4087-88d4-a36b2d308c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:33.517775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144952-b95f-4087-88d4-a36b2d308c63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:33.536090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0144952-b95f-4087-88d4-a36b2d308c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b0144952-b95f-4087-88d4-a36b2d308c63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:33.541923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:33.543375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:33.544837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 204/390] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:33.553054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:33.681276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f707103-4a08-47f4-a191-1bea220201dd None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:33.681564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f707103-4a08-47f4-a191-1bea220201dd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:33.681938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f707103-4a08-47f4-a191-1bea220201dd None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:33.683463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 205/391] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:33.688179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:33.688490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:33.688729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:33.709076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:33.709690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:33.711012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:33.735796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f35d6-d39e-4d09-bd2b-96e2ad634c64 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:33.792741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:33.793377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:33.793657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:33.793928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:33.794647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:33.794948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:33.798311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:33.811647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 187/392] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:33.821463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:33.845708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 206/393] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:33.943923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ca57a-e085-46ba-8923-fad57cf16cd0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:33.990577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046f7b1-df49-46ca-a070-224970b5accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:33.991977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046f7b1-df49-46ca-a070-224970b5accd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:33.992444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046f7b1-df49-46ca-a070-224970b5accd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:33.992919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046f7b1-df49-46ca-a070-224970b5accd None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:33.998300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046f7b1-df49-46ca-a070-224970b5accd None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:34.000241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046f7b1-df49-46ca-a070-224970b5accd None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:34.002056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 188/394] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:33 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:34.007647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.010037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c10d7-a551-4f52-83db-2b78f758805f None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.010536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c10d7-a551-4f52-83db-2b78f758805f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.011013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c10d7-a551-4f52-83db-2b78f758805f None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.012507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 189/395] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:34.017905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914c2ea-6332-46b3-a188-54b08f19f658 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.018522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914c2ea-6332-46b3-a188-54b08f19f658 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.019008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914c2ea-6332-46b3-a188-54b08f19f658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.058226 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d914c2ea-6332-46b3-a188-54b08f19f658 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:34.059458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d914c2ea-6332-46b3-a188-54b08f19f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:34.082388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:34.176033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 190/396] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 24 04:47:34.176233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:34.178205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5e9c5-9d57-4eec-829e-1e47743ce9b6 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.178754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5e9c5-9d57-4eec-829e-1e47743ce9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.179307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5e9c5-9d57-4eec-829e-1e47743ce9b6 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.180724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 191/397] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:34.185197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7b88ed-b9da-4402-8a19-a5b3fba1ae8d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.186322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5e9c5-9d57-4eec-829e-1e47743ce9b6 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.220712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac34be-57c1-4a3f-bbeb-e570d9f36751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.221359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac34be-57c1-4a3f-bbeb-e570d9f36751 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.221641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac34be-57c1-4a3f-bbeb-e570d9f36751 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.221916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac34be-57c1-4a3f-bbeb-e570d9f36751 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.223141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac34be-57c1-4a3f-bbeb-e570d9f36751 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:34.224714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac34be-57c1-4a3f-bbeb-e570d9f36751 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:34.226512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 207/398] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:34.227623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25612766-0096-4f04-b966-c327023e60c1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.227907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25612766-0096-4f04-b966-c327023e60c1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.228179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25612766-0096-4f04-b966-c327023e60c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.250519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-25612766-0096-4f04-b966-c327023e60c1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:34.251714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-25612766-0096-4f04-b966-c327023e60c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:34.323464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 208/399] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:34.325171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25612766-0096-4f04-b966-c327023e60c1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.336648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5e9c5-9d57-4eec-829e-1e47743ce9b6 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.405300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.405938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.406254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.406525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.425418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:34.432033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:34.432888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:34.434078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 192/400] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:34.441613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.451827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:34.485394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:34.494065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25612766-0096-4f04-b966-c327023e60c1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.533145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.534127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.534480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.534875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573/groups/dc46ba8ff0d34d51bee067470f850611/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.536850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] RBAC: Authorizing `identity:create_grant(project_id=595b8e1093aa49399a51e159588da573, group_id=dc46ba8ff0d34d51bee067470f850611, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:34.538396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:34.558055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 209/401] 10.222.0.50 () {58 vars in 1442 bytes} [Sat Sep 24 04:47:34 2022] PUT /identity/v3/projects/595b8e1093aa49399a51e159588da573/groups/dc46ba8ff0d34d51bee067470f850611/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:34.558055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.566203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac64d4e-202d-4c1e-8464-11bc910cadb5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.630367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.631408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.631886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.632381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.633398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:34.633928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:34.635679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:34.656136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 193/402] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:34.666643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.685716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e906a-eb0d-47eb-8272-f1a9b906512f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.726284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.726935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.727216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.727489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.740640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:34.741863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:34.747364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:34.748326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:34.749288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 210/403] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:34.750429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.768430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:34.785301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f9fe4-d964-460b-be1f-5247a6813441 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.823641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.824559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.824981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.825389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.837788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:34.843942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:34.844974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:34.845947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:34.852668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 194/404] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:34.853874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:34.871052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81873b-57d9-4126-a828-1090a300cbdb None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:34.910565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:34.911487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:34.911954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:34.912393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:34.914303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:34.916390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:34.918235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 211/405] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:34.919240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.003694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59252c-b8c9-4d5e-9fd1-8acc09b00fc6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.024988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.039992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.049189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d45d3-231b-42d9-87df-55ca86303425 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.053026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.053712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.054094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.054504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.055281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:35.055698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.057231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.069156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 195/406] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.070175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.089260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.090244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.090693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.091275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.092512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:35.093144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.094849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.105934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 212/407] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.111007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.186446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a0c4f-c71e-44dc-94ed-9f43c53b1c34 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.221058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.222061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.222531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.222992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.224344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.225965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.233265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 196/408] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:35 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.234649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.235889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec78ab7-bea3-4309-9745-089932e231f9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.281842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.282468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.282761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.283037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.284408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.285953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.287922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 213/409] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:35 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.289021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.301238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.306060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.356620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800532c-0091-4602-a062-b2247d96fc2b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.393747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.394418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.394699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.394975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] PATH_INFO: `/v3/groups/e01927bc54e044cba76b0a9d54171881` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.396095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] RBAC: Authorizing `identity:get_group(group_id=e01927bc54e044cba76b0a9d54171881)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.397542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.399456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 197/410] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:35 2022] GET /identity/v3/groups/e01927bc54e044cba76b0a9d54171881 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.400222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.415465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ddf31-32ea-4f43-aba5-a24ea83382d4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.452532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3c2f94-1c45-48c6-96ea-999a2ac35787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.453339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c2f94-1c45-48c6-96ea-999a2ac35787 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.453831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c2f94-1c45-48c6-96ea-999a2ac35787 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.454361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3c2f94-1c45-48c6-96ea-999a2ac35787 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537/groups/dc46ba8ff0d34d51bee067470f850611/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.457021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c2f94-1c45-48c6-96ea-999a2ac35787 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1259daa670c46fcacf1d9b1449f2537, group_id=dc46ba8ff0d34d51bee067470f850611, role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.459252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3c2f94-1c45-48c6-96ea-999a2ac35787 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.473526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 214/411] 10.222.0.50 () {58 vars in 1442 bytes} [Sat Sep 24 04:47:35 2022] PUT /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537/groups/dc46ba8ff0d34d51bee067470f850611/roles/399f7b70c70b4f09addcf156bf604eab => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:35.475055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406938-a932-4b9c-9a8b-122acb8f36e0 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.475348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406938-a932-4b9c-9a8b-122acb8f36e0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.475619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89406938-a932-4b9c-9a8b-122acb8f36e0 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.476874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 215/412] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:35.478417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89406938-a932-4b9c-9a8b-122acb8f36e0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.522921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92b43e9-392e-46db-9ce8-85740d4f3e85 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.555440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.569887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c067f02-9192-43fa-b0bd-100f41a88dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.571006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067f02-9192-43fa-b0bd-100f41a88dfb None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.571624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067f02-9192-43fa-b0bd-100f41a88dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.572205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c067f02-9192-43fa-b0bd-100f41a88dfb None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.573936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c067f02-9192-43fa-b0bd-100f41a88dfb None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.575725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c067f02-9192-43fa-b0bd-100f41a88dfb None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.577975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 198/413] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:35 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.579676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.580205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.580694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.581170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.606154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:35.607821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:35.622681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89406938-a932-4b9c-9a8b-122acb8f36e0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.671248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.671884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.672161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.672432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] PATH_INFO: `/v3/projects/f9c9e5a50c664c33aa419611e20c9c00` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.673908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c9e5a50c664c33aa419611e20c9c00)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.675698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.677461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 216/414] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:35 2022] GET /identity/v3/projects/f9c9e5a50c664c33aa419611e20c9c00 => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:35.678136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.712269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 199/415] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:35.714753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:35.787996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295aeeb6-5968-42a1-b33d-9a081e395027 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.823335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.823918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.824205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.824469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537/groups/dc46ba8ff0d34d51bee067470f850611/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.826518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1259daa670c46fcacf1d9b1449f2537, group_id=dc46ba8ff0d34d51bee067470f850611, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.828150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.856508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.862585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 217/416] 10.222.0.50 () {58 vars in 1442 bytes} [Sat Sep 24 04:47:35 2022] PUT /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537/groups/dc46ba8ff0d34d51bee067470f850611/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:35.863567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03d6d1-8dd3-4a7f-a479-d97c9215f628 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:35.893604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:35.909631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:35.911074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:35.911594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:35.912092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] PATH_INFO: `/v3/projects/f9c9e5a50c664c33aa419611e20c9c00/groups/e01927bc54e044cba76b0a9d54171881/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:35.914597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9c9e5a50c664c33aa419611e20c9c00, group_id=e01927bc54e044cba76b0a9d54171881, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:35.917049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:35.939474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 200/417] 10.222.0.50 () {58 vars in 1442 bytes} [Sat Sep 24 04:47:35 2022] PUT /identity/v3/projects/f9c9e5a50c664c33aa419611e20c9c00/groups/e01927bc54e044cba76b0a9d54171881/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:36.054793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:36.175764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ee79-48ed-4247-9425-c6a85ee102b1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:36.215854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:36.216670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:36.217046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:36.217422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:36.231632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:36.237851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:36.238782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:36.239565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:36.241917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 218/418] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:36.248566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:36.382534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd61030-0766-441c-9b03-ce347bd020f1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:36.433497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:36.434049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:36.434325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:36.434587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:36.435248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:36.435541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:36.436888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:36.450027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 201/419] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:36.455815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:36.545547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:36.604636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21046aa-f29f-4bb0-b597-3d7d50e632db None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:36.652428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:36.653055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:36.653333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:36.653604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:36.737613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:36.739137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:36.802488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:36.851146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 219/420] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:36.856564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:36.977056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:37.046022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c1397-451b-4897-b0a3-7211e79b1ea9 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:37.095668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768a8b1-26d4-45c7-99c4-db9e09d65364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:37.096323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768a8b1-26d4-45c7-99c4-db9e09d65364 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:37.096608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768a8b1-26d4-45c7-99c4-db9e09d65364 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:37.096884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768a8b1-26d4-45c7-99c4-db9e09d65364 None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:37.098076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768a8b1-26d4-45c7-99c4-db9e09d65364 None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:37.099633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768a8b1-26d4-45c7-99c4-db9e09d65364 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:37.101591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 202/421] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:36 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:37.107748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:37.141847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730482-7eb3-4a62-b513-2c333c6cdac2 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:37.142161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730482-7eb3-4a62-b513-2c333c6cdac2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:37.142435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68730482-7eb3-4a62-b513-2c333c6cdac2 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:37.144074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 203/422] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:37.149226 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:37.149515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:37.149793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:37.168972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:37.170199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:37.270341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaa343-2061-40da-920d-b3e52aac42b3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:37.300318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 204/423] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:37.324693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:37.325399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:37.325720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:37.326051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:37.327274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:37.328704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:37.330609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 220/424] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:37 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:37.336247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:37.396190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:37.462612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723677b-7f0d-4342-83fe-f9976bfcf4f2 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:37.510472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:37.511094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:37.511373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:37.511691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:37.512432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=595b8e1093aa49399a51e159588da573) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:37.512736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:37.514222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:37.522844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 205/425] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:37 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=399f7b70c70b4f09addcf156bf604eab => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:37.656982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:37.701557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:37.788928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361dea8-8624-4f24-bd6e-74c0677d67f6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:37.823602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:37.824663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:37.825206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:37.825668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:37.836299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:37.841875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:37.842794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:37.843594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:37.846315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 221/426] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:37.852142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:37.959495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001d883-960d-43fd-ab43-28404bf9c9ec None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:37.972342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:38.005116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:38.005779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.006111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.006432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.007117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:38.007514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:38.008829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd16960-8bf8-4027-ac93-cc19dca82c93 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:38.018109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 206/427] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:38.024277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:38.128916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:38.147236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c98ed-375f-4bdf-9736-a3ec00214043 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:38.184314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:38.184949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.185227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.185549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.223766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bf306-1b6b-4cd1-97c5-0b6bbd66a688 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.223766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bf306-1b6b-4cd1-97c5-0b6bbd66a688 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.223766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bf306-1b6b-4cd1-97c5-0b6bbd66a688 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.229331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 207/428] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:38.230668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.231008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.231286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.250997 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:38.252343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:38.264044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:38.266714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:38.329052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 208/429] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:38.369124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:38.386824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 222/430] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:47:38.392453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:38.496756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:38.500657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a220f-2334-4107-bc75-26f54d42a849 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:38.535440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac20e9-ede5-4163-b02d-8b7dde7a30ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:38.536077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac20e9-ede5-4163-b02d-8b7dde7a30ba None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.536358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac20e9-ede5-4163-b02d-8b7dde7a30ba None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.536631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac20e9-ede5-4163-b02d-8b7dde7a30ba None admin] PATH_INFO: `/v3/projects/595b8e1093aa49399a51e159588da573` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.537757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac20e9-ede5-4163-b02d-8b7dde7a30ba None admin] RBAC: Authorizing `identity:get_project(project_id=595b8e1093aa49399a51e159588da573)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:38.539218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac20e9-ede5-4163-b02d-8b7dde7a30ba None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:38.541293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 209/431] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:38 2022] GET /identity/v3/projects/595b8e1093aa49399a51e159588da573 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:38.546719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:38.565742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2bced-70dd-40f5-b916-5ea5701a836e None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.566055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2bced-70dd-40f5-b916-5ea5701a836e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.566321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2bced-70dd-40f5-b916-5ea5701a836e None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.567894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 210/432] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:47:38.573091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.573373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.573643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.596329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:38.597736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:38.671540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213308b-e6d5-4c70-ad88-05d24c3ae352 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:38.675930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 211/433] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:38.677011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:38.713634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:38.714590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.715042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.715482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.717339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:38.719236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:38.726287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 223/434] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:38 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:38.727738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:38.790280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:38.803282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:38.810598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e95b531-78e1-494c-b8a2-fb80d9483fa1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:38.839859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d20c4-ae77-4d56-b93b-ba545675f8e0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:38.848666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116da8cb-8d58-439e-9134-5a071dc044d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:38.849647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116da8cb-8d58-439e-9134-5a071dc044d9 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.849927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116da8cb-8d58-439e-9134-5a071dc044d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.850548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116da8cb-8d58-439e-9134-5a071dc044d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.861749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116da8cb-8d58-439e-9134-5a071dc044d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-116da8cb-8d58-439e-9134-5a071dc044d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:38.866624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:38.867539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:38.868331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 212/435] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:38.868988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbf034-08a4-453c-9026-24718389ef77 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.869054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbf034-08a4-453c-9026-24718389ef77 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.869728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbf034-08a4-453c-9026-24718389ef77 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.870745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 213/436] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:38.875870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbf034-08a4-453c-9026-24718389ef77 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:38.879958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:38.880565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.880847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.881128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.881847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=6768099ea599447f9de57814f623441c, scope.project.id=595b8e1093aa49399a51e159588da573) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:38.882185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:38.883616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8efdb0-8fb2-48fc-a75f-b1e8ee9e2c42 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:38.897289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 224/437] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:38 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=595b8e1093aa49399a51e159588da573&role.id=6768099ea599447f9de57814f623441c => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:38.899776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:38.900750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:38.901290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:38.927644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:38.930068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:38.993744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbf034-08a4-453c-9026-24718389ef77 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.026175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 225/438] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:39.026175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.042783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.043313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.043607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.043858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.044539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:39.044832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.046331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.056312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 214/439] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:39.061503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.092392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.102870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.169276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f81bd6-12ac-4027-b570-4fa0f6bda973 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.214808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd80a2-8d52-45df-ad37-6a168c26a673 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.220468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.221290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.221573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.221851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.236097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:39.241066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:39.241945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 24 04:47:39.242807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:39.246891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 226/440] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:39.248869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.263929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.264590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.264817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.265085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.344409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.345503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.367740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.380966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fabea-c9e4-433a-b722-19eedfae35ce None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.391219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.422859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.423475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.423755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.424028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.424693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] RBAC: Adding query filter params (name=member) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:39.424995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.426413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.438482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 227/441] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:39.439311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.464094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 215/442] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 24 04:47:39.464965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.553724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a17b95-a767-4b7c-91a3-90c950b2e854 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.565697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecede2ed-0f51-4bc1-a693-0fa03a6a59ed None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.582685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.586695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.587310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.587589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.587856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.598638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:39.600258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.600855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.601125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.601386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:39.603997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:39.604957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:39.605731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 228/443] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:39.607412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.637440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.672372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.673820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.710496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe9379-aace-47eb-a1d0-c63da5effdb8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.745525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.746184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.746469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.746745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.748228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.750398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.752109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 229/444] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:39.752861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.773233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 216/445] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:39.774077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.792039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.852518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:39.856900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb0e38-9e29-4a92-aa99-d262b88ea699 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.878607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8874ff-ddaf-4965-adba-b5270ef79c4f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:39.892046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.892646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.892921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.893187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.893831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:39.894154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.895511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.903716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 230/446] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:39.904499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:39.917151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:39.917771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:39.918078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:39.918354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:39.919442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:39.920868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:39.922440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 217/447] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:39.923184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.004619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b452d3c-cfe4-4724-878d-ec55d7bc6e2b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.006592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:40.021402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c2e1e-a848-48c9-9d0c-f8b5d0b23300 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.043310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.043926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.044201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.044469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] PATH_INFO: `/v3/projects/a1259daa670c46fcacf1d9b1449f2537` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.045566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=a1259daa670c46fcacf1d9b1449f2537)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.047056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.048862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 231/448] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/projects/a1259daa670c46fcacf1d9b1449f2537 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:40.049415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.058983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.059606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.059883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.060156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.076629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:40.129419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.130844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.147875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ce5e9-1d30-424b-a0cc-09639ef18ec4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.183681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.184372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.184516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.184791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.185516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=a1259daa670c46fcacf1d9b1449f2537) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:40.185827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.187590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.193504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 232/449] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:40 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=399f7b70c70b4f09addcf156bf604eab => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:40.194118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.232864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:40.252443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 218/450] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:40.258542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.312389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:40.324464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bde0c-170a-464e-806a-9a07aca732eb None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.364280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.364915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.365191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.365465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] PATH_INFO: `/v3/groups/dc46ba8ff0d34d51bee067470f850611` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.366702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] RBAC: Authorizing `identity:get_group(group_id=dc46ba8ff0d34d51bee067470f850611)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.368572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.370708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 233/451] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:40 2022] GET /identity/v3/groups/dc46ba8ff0d34d51bee067470f850611 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:40.374041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63018792-e70d-4a42-8920-8c701bc0ba4b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.376370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.418314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.418907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.419188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.419459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] PATH_INFO: `/v3/projects/f9c9e5a50c664c33aa419611e20c9c00` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.421101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] RBAC: Authorizing `identity:get_project(project_id=f9c9e5a50c664c33aa419611e20c9c00)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.423028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.425192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 219/452] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:47:40 2022] GET /identity/v3/projects/f9c9e5a50c664c33aa419611e20c9c00 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:40.430561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.482076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258f884-1c8e-44bd-a8c9-1c3a58b1168c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.486369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:40.515017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.515615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.515891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.516162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.516881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] RBAC: Adding query filter params (group.id=dc46ba8ff0d34d51bee067470f850611, role.id=6768099ea599447f9de57814f623441c, scope.project.id=a1259daa670c46fcacf1d9b1449f2537) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:40.517183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.518649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.524223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 234/453] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:40 2022] GET /identity/v3/role_assignments?group.id=dc46ba8ff0d34d51bee067470f850611&scope.project.id=a1259daa670c46fcacf1d9b1449f2537&role.id=6768099ea599447f9de57814f623441c => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:40.535697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e88b67-0275-4c56-95ad-313f87147c86 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.573528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97106e-eabe-48a3-bf15-28f0626279af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.574200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97106e-eabe-48a3-bf15-28f0626279af None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.574480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97106e-eabe-48a3-bf15-28f0626279af None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.574752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97106e-eabe-48a3-bf15-28f0626279af None admin] PATH_INFO: `/v3/groups/e01927bc54e044cba76b0a9d54171881` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.575937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97106e-eabe-48a3-bf15-28f0626279af None admin] RBAC: Authorizing `identity:get_group(group_id=e01927bc54e044cba76b0a9d54171881)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.577401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97106e-eabe-48a3-bf15-28f0626279af None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.579079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 220/454] 10.222.0.50 () {56 vars in 1180 bytes} [Sat Sep 24 04:47:40 2022] GET /identity/v3/groups/e01927bc54e044cba76b0a9d54171881 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:40.585398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:40.613178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:40.696544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27c1b7-4bf9-41b7-9d5f-47c2ed002157 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:40.734962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:40.735594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:40.735872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:40.736142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:40.736864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] RBAC: Adding query filter params (group.id=e01927bc54e044cba76b0a9d54171881, role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=f9c9e5a50c664c33aa419611e20c9c00) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:40.737160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:40.738654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998113d9-96a2-4d7c-85ae-522227cbd88c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:40.744869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 235/455] 10.222.0.50 () {56 vars in 1376 bytes} [Sat Sep 24 04:47:40 2022] GET /identity/v3/role_assignments?group.id=e01927bc54e044cba76b0a9d54171881&scope.project.id=f9c9e5a50c664c33aa419611e20c9c00&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:43.531936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b99e2-8362-4765-ac99-82460fdc9e3b None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:43.532961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b99e2-8362-4765-ac99-82460fdc9e3b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:43.533420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b99e2-8362-4765-ac99-82460fdc9e3b None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:43.535392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 221/456] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:43.540890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314900f3-399e-43b3-b437-83a22fccfb17 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:43.542338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314900f3-399e-43b3-b437-83a22fccfb17 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:43.542817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314900f3-399e-43b3-b437-83a22fccfb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:43.571000 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-314900f3-399e-43b3-b437-83a22fccfb17 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:43.572515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-314900f3-399e-43b3-b437-83a22fccfb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:43.667508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65f19-2ba5-4d63-a253-0b8518614cb9 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:43.668210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65f19-2ba5-4d63-a253-0b8518614cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:43.668494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 236/457] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:43.668897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65f19-2ba5-4d63-a253-0b8518614cb9 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:43.670638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 222/458] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:43.688096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc81c-1836-4306-8261-514c81663f99 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:43.688663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc81c-1836-4306-8261-514c81663f99 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:43.689139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30dc81c-1836-4306-8261-514c81663f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:43.710990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d30dc81c-1836-4306-8261-514c81663f99 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:43.712404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d30dc81c-1836-4306-8261-514c81663f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:43.726479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:43.803425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 237/459] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:43.827011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e13bd-0725-4540-8623-76747b791bb7 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:43.827600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e13bd-0725-4540-8623-76747b791bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:43.828057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e13bd-0725-4540-8623-76747b791bb7 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:43.829856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 223/460] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:43.834895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:43.835182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:43.835458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:43.863069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:43.864261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:43.951484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 238/461] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:43.986695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e13bd-0725-4540-8623-76747b791bb7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.111526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.131284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e13bd-0725-4540-8623-76747b791bb7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.170707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:44.195756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.196407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.196690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.196966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.204542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:44.220004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.221450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.224916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf7806-c155-4a8f-a54b-0893a2d7376a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:44.251969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:44.252946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:44.253789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:44.253789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:44.253789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:44.253789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 224/462] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:44.259146 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.283557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.284551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.285020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.285478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.309857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.311673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:44.337273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:44.338871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:44.339715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:44.339715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:44.339715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:44.341840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 239/463] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:44.342889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.408466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8adae-fa44-48ce-a0b1-51f0750f5e1b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.444881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.445476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.445749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.446039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.446683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:44.446979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.448972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.451973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffe563-cc3d-4324-8267-ffa4812f6d5f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.468930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 225/464] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:44.470482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.481604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:44.489118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.489746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.490042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.490513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.501736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.503203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.507611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6da9db09-40f8-4625-b495-6e473023a9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:44.517809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:44.518747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:44.519599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:44.519599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:44.519599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:44.519599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 240/465] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:44.520631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ff98-ea9a-4d80-98f4-3c45a5bd545e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.520631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ff98-ea9a-4d80-98f4-3c45a5bd545e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.520631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999ff98-ea9a-4d80-98f4-3c45a5bd545e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.526415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 241/466] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:44.526415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999ff98-ea9a-4d80-98f4-3c45a5bd545e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.584882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a69a0-25f7-4249-90f4-ff0de0a7e33a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.619572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.620172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.620446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.620723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.621362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:44.621657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.623073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.641735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 226/467] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:44.641735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.656972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999ff98-ea9a-4d80-98f4-3c45a5bd545e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.696428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0dbe6-292e-4b6b-a027-fccd6d895871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.696924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0dbe6-292e-4b6b-a027-fccd6d895871 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.697585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0dbe6-292e-4b6b-a027-fccd6d895871 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.697856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0dbe6-292e-4b6b-a027-fccd6d895871 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.698853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0dbe6-292e-4b6b-a027-fccd6d895871 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.700315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0dbe6-292e-4b6b-a027-fccd6d895871 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.725117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 242/468] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:44 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:44.727120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.727619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.728347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.755849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:44.758056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:44.764630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:44.778051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c816c153-ac27-4992-9b27-6426708b3bff None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:44.796303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:44.819948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:44.821056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:44.821587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:44.822126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:44.823516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:44.824124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:44.825914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:44.843247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 227/469] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:44.844688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.848509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 243/470] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:44.849950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:44.964812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a893ec-2fc1-4534-9963-0b2dafe1c4d6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:45.004020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e30def-a675-4f8f-9c2b-a89ac15804e6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:45.016946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a35ac5-a3bf-468c-ae74-728cd48a5435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:45.017564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35ac5-a3bf-468c-ae74-728cd48a5435 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.018293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35ac5-a3bf-468c-ae74-728cd48a5435 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.018572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a35ac5-a3bf-468c-ae74-728cd48a5435 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.019396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a35ac5-a3bf-468c-ae74-728cd48a5435 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:45.020996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a35ac5-a3bf-468c-ae74-728cd48a5435 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:45.050702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 244/471] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:44 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:45.051501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:45.051987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86a005-00af-4b44-bdc0-0797428c9e34 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.052317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86a005-00af-4b44-bdc0-0797428c9e34 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.052601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a86a005-00af-4b44-bdc0-0797428c9e34 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.052963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.053443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.053837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 245/472] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:45.054125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.054921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:45.056573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:45.059930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.060301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.060770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.083976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:45.085456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:45.085885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 228/473] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:44 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:45.108572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:45.179757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 246/474] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:45.195488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:45.222021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:45.347042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed39a3-af90-4740-a966-4dba2800e2f5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:45.386473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:45.387106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.387390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.387653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.398573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:45.400039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:45.422319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:45.423266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:45.424108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:45.424108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:45.424108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:45.424108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 229/475] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:45.429524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:45.487264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:45.555167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b4ea2b-ab9a-47f1-ba5b-1af0e790ff96 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:45.572980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:45.607681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:45.608599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.609073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.609496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.610176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:45.610476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:45.611906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:45.622935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973eee4-ee0f-43d5-99c7-91d8a3df65d7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:45.640992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 247/476] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:45.648756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:45.684450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:45.685634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.687133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.687730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:45.688894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.690680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:45.693891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:45.711722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.sql.core [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 04:47:45.711722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 04:47:45.711722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [parameters: {'id': 'a654a181c7a744d8a1535222a46e2d64', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 04:47:45.711722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81393) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application conn.query(q) Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application result.read() Sep 24 04:47:45.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return ref.to_dict() Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 04:47:45.744896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application next(self.gen) Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application yield resource Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application next(self.gen) Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise self.value Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application session.commit() Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 04:47:45.745796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self.session.flush() Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application compat.raise_( Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 04:47:45.746660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 04:47:45.747534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application conn.query(q) Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application result.read() Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 04:47:45.748341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [parameters: {'id': 'a654a181c7a744d8a1535222a46e2d64', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:45.749121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 24 04:47:45.752501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:45.764769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 230/477] 10.222.0.50 () {60 vars in 1131 bytes} [Sat Sep 24 04:47:45 2022] POST /identity/v3/roles => generated 144 bytes in 272 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:45.770551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:45.865433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda34d1-5ee4-4ce2-a798-d8484eb6843a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:45.902537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:45.903691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:45.904193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:45.904666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:45.905559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:45.907301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:45.939127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 248/478] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:45 2022] POST /identity/v3/users => generated 249 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:45.968311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69697b25-3e53-4a02-bb20-e84a96c67606 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:46.004625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:46.014378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df445ba4-e6b8-4c3b-b102-d63b4ca1a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:46.019293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df445ba4-e6b8-4c3b-b102-d63b4ca1a3ad None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:46.019910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df445ba4-e6b8-4c3b-b102-d63b4ca1a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:46.020386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df445ba4-e6b8-4c3b-b102-d63b4ca1a3ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:46.035541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df445ba4-e6b8-4c3b-b102-d63b4ca1a3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-df445ba4-e6b8-4c3b-b102-d63b4ca1a3ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:46.041781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:46.042888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 24 04:47:46.043715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:46.046439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 231/479] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:47:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:46.052732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:46.137132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:46.185342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0ca1ca-6b47-4102-ad82-fb438d2119ad None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:46.222617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:46.223310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:46.223593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:46.223867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:46.224540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:46.224840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:46.226284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ba8a3-dba6-4c1e-892b-550c308e03a2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:46.235835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 249/480] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:47.450292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeb6d0-906d-4ba5-abc5-b14be0ca52f4 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:47.450805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeb6d0-906d-4ba5-abc5-b14be0ca52f4 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:47.450805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaeb6d0-906d-4ba5-abc5-b14be0ca52f4 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:47.452426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 232/481] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:47.467149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954d4ad-a2a2-4fe8-8a5c-2b46ecfac9ae None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:47.467612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954d4ad-a2a2-4fe8-8a5c-2b46ecfac9ae None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:47.467683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c954d4ad-a2a2-4fe8-8a5c-2b46ecfac9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:47.490984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c954d4ad-a2a2-4fe8-8a5c-2b46ecfac9ae None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:47.492192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c954d4ad-a2a2-4fe8-8a5c-2b46ecfac9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:47.576641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 250/482] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:47.898453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c263d-703b-4aff-b3d2-1a28e9fbd662 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:47.899129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c263d-703b-4aff-b3d2-1a28e9fbd662 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:47.899607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c263d-703b-4aff-b3d2-1a28e9fbd662 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:47.901471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 233/483] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:47.907081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:47.907607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:47.908073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:47.931192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:47.932217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923d668-9c58-4a87-af40-7f353e4be3c8 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:47.932500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923d668-9c58-4a87-af40-7f353e4be3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:47.932767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923d668-9c58-4a87-af40-7f353e4be3c8 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:47.933383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:47.934096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 234/484] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:47.938863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:47.939142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:47.939411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:47.959517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:47.960685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:48.015888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.026679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 251/485] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:48.027594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.051992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 235/486] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:48.155398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbc3dc-0fb6-41d3-90f8-9b8102c61db3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.189288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.189913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.190221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.190492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.200498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:48.205249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:48.206170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:48.206968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 252/487] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:48.212120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.344333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468d5a9-4399-4592-80ea-de0c24c209cf None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.356852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.383636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.425677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.426228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.426502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.426765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.427429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:48.427720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:48.429096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:48.441393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 236/488] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:48.447080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.448277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.496943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d99691-c79e-4e5b-a060-831bd1771b25 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.534035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.534035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.534035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.534035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.547363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:48.552444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:48.553328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:48.554150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:48.554825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 253/489] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:48.560317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.575815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ad3de-01db-48f6-99eb-77f18abe55c0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.619165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.620305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.620580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.620845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.635529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:48.637261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:48.661071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:48.662183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:48.663079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:48.663079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:48.663079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:48.663079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 237/490] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:48.663288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.682527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80a1a0-1641-4732-8302-0468cd6e9532 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.699586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.707256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.727938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.729374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.729794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.730265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.744569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:48.751097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:48.752054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:48.752858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:48.760640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 254/491] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:48.760640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.799290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72aee8b-537f-4638-81e9-5ad539599fc0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.842403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.843376 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.843646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.843921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.844937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:48.845251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:48.846641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:48.855132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 238/492] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:48.855843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.869766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590500e-d874-4df6-89f7-785c0614bce1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:48.906704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:48.907332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.907610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.907882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.908547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:48.908851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:48.910318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ed472-ff81-4755-bb4f-753ceb34b50c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:48.923528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 255/493] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:48.924670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896c730-9958-40be-99b3-6b8074bbdcff None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:48.924958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896c730-9958-40be-99b3-6b8074bbdcff None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:48.925238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896c730-9958-40be-99b3-6b8074bbdcff None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:48.930834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 256/494] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:48.930834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896c730-9958-40be-99b3-6b8074bbdcff None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:48.963563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.969400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:48.973715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfb53b-9afa-46a8-929f-76452cf11f0f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.008403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.009004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.009287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.009559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.010826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:49.011124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.012499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.021319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 239/495] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:49.023282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.045241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896c730-9958-40be-99b3-6b8074bbdcff None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.081269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.081885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.082184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.082455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.092438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.093874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-c3c36c4e-84f0-4e06-b97f-e4906582d0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:49.108302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:49.109196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:49.110064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:49.110064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:49.110064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:49.110064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 257/496] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:49.110520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef381a1-c037-496e-9153-bcbe4a16e2a2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.110794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef381a1-c037-496e-9153-bcbe4a16e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.111067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef381a1-c037-496e-9153-bcbe4a16e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.127383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e83e8-43cc-4bba-b63c-cd6c26d43ad2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.128870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-bef381a1-c037-496e-9153-bcbe4a16e2a2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:49.130071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-bef381a1-c037-496e-9153-bcbe4a16e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:49.160035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.160651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.160935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.161207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.168924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.170396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:49.182597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:49.183524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:49.184421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 240/497] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:49.185082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.197354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 258/498] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:49.198613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215cc3e-f596-4f1e-b10b-4a2911d86d29 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.198891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215cc3e-f596-4f1e-b10b-4a2911d86d29 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.199159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215cc3e-f596-4f1e-b10b-4a2911d86d29 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.200508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 259/499] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:49.201244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.201351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215cc3e-f596-4f1e-b10b-4a2911d86d29 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.223379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.300623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ba015-8fc4-4200-b07c-d0986c46c42e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.316896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215cc3e-f596-4f1e-b10b-4a2911d86d29 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.334770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.335234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.335510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.335778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.346801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.348255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.352911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.353513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.354441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.354715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.355811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:49.356121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.357869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29d560f-1e2e-4304-9c43-4813101564f7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:49.362795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:49.363649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:49.364471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:49.364471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:49.364471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:49.364471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 241/500] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:49.364690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.379550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 260/501] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:49.379550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.379550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.379550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.394722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:49.395889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:49.441249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.474110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 261/502] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:49.474110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.475306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.498294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8482ce3-659c-4e8a-8bbc-d1acd939a0ab None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.534845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.535450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.535730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.536006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.536735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:49.537128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.538511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.574556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0213885-2033-4ba7-8a9b-806f9a535dfc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.577130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 242/503] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:49.578411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.613794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.614422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.614696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.614966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.615602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:49.615894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.617265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.633802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 262/504] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:49.633802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.689741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5ffa7-98d4-4c6f-8076-83c30ae7cb9b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.702338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.714392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.726894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.727564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.727912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.728252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.735985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.737511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.743709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb2e8-4a1e-42e4-941e-9910b8a20914 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:49.749230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:49.750592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:49.751505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 243/505] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:49.752055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.779151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.779770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.780042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.780310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.789916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:49.795708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:49.796748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 263/506] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:49.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:49.854882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab4523-cb97-4f34-8dbc-fe3c8dae3fd5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.889628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.890259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.890537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.890804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.899016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70061f-533a-421f-b421-a597d1a4a42b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:49.920813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.931362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:49.932494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:49.932765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:49.933042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:49.940330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.941259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:49.941762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.958197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:49.958354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:49.959245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 04:47:49.964972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:49.965593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:49.965684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 264/507] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:49.965684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.060930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e6a15-b4ab-4e1d-9e5d-ea7ed637468d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.062046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 244/508] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:50.062883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.098405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.099029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.099308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.099580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.110368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.111809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:50.126379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:50.127265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:50.128228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:50.128228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:50.128228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:50.128228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 265/509] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:50.128637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.145160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.159959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deee30e-5389-4c73-8352-558a2a18ceeb None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.163506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.193074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.193674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.193949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.194246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.194969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:50.195269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.196684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.213087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 245/510] 10.222.0.50 () {56 vars in 1137 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:50.213871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.230760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e745-e9f9-42e9-8362-c89d5ee67b0b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.263343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.263935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.264208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.264476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.265119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:50.265416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.266813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.274352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 266/511] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:50.275108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.310849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282919d6-99d1-4acc-88fd-5ad3028e365e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.344319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.344854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.345129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.345399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.346155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:50.346456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.348250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.362425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.369084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.370399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 246/512] 10.222.0.50 () {56 vars in 1137 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:50.371769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.374013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b01a3-d233-4823-a4d9-b0b37ccb7dee None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.407881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.408526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.408811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.409081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.419507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.420947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:50.435322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:50.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:50.437062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:50.437062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:50.437062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:50.437062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 267/513] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:50.437284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.477290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695fa8ff-e113-4115-b069-6fc2c16e015f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.511661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.512257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.512536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.512808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.513455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:50.513754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.515149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.526752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 247/514] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:50.527539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.538118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e959db-e753-42a5-86d6-fa681c571fac None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.571087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.571690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.572136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.572412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.584931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.590948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.625832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03ebf7-e4f8-4243-b8fe-c57cb5b4d7b3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.639939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.641389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.661698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.662361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.662639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.662913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.674560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.676007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:50.690619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:50.691602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:50.692447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:50.692447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:50.692447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:50.692447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 248/515] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:50.692837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.747772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 268/516] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:50.748605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.796982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.800098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7184ec88-3877-447d-94b7-6918e6825cec None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.812545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:50.835184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.835781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.836061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.836325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.846164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f348b0-f7e8-4128-9542-58b4c83d68d4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:50.879046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:50.879575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:50.879852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:50.880123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:50.880750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:50.881047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.882453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.893858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 269/517] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:50.894669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:50.902727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:50.904156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:50.993465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccdbd0-1e91-442a-9b11-140178725da9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.000231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 249/518] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:51.001311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.004100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.015681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.029006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.029610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.029890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.030259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.030902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.032332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.053012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 270/519] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:47:50 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:47:51.053985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.124188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd3277-38f7-4459-a3c1-07202c69c5e9 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.162714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323fa67-7805-448f-89e7-d7a67f43acf6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.163761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.164501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.164806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.165163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.165829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:51.166195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.167588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.181096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 250/520] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:51.182928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.206249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.206903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.207181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.207453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.219754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.221218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:51.238045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:51.238958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:51.239809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:51.239809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:51.239809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:51.239809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 271/521] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:51.240031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.264887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.267771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.302673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f8611-8f0c-4e31-8a7a-7ad3ae0e4d77 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.339438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.340434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.340777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.341109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.350102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.352077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.354598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f964a32-916b-4199-8437-1fb6e6af1a71 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:51.366774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:51.367851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:51.368799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 251/522] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:51.369979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.394539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.395170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.395447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.395721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.407322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.408782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:51.427190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:51.428279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:51.429266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:51.429266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:51.429266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:51.430703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 272/523] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:51.431595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.485392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f50e9ce-3417-4db2-a677-e45b426e0c86 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.506440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.517690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.524440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.525220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.525517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.525847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.533764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.535466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.540464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02faf1-ff17-4f86-942b-1550ddefc23b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:51.547604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:51.553424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 252/524] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:51.554594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.584949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.585795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.586108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.586423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.587132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:51.587440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.588902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.605949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 273/525] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:51.605949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.669964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd19e99-8d1d-4b25-8622-f70d0671a6bf None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.704826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.705495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.705872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.706286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.707079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:51.707482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.709021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.717951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87a661-91ee-41af-bcb9-8ea991270f4f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.732451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 253/526] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:51.732451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.755169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.763977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.764741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.765004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.765586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.766268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:51.766566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.767962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.779594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 274/527] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:51.780220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.800465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:51.850425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c65f8-8561-460b-8c67-8fa2cf599fae None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.887578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e54f761-72e1-4212-930d-0f9e5e41b341 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:51.887766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.888358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.888631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.888894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.889613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:51.889907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.891361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.913744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 254/528] 10.222.0.50 () {56 vars in 1139 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:51.913744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:51.924362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:51.925117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:51.925402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:51.925679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:51.932858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:51.934331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:51.946117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:51.946952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:51.947860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 275/529] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:51.948667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.007153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.061570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485dd0d5-00d8-4db7-bcac-f100d74997b9 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.066185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5b063-48be-4b1f-9e33-16e353098274 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.080612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.107861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.108639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.109042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.109335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.113956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.114837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.115337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.116058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.116446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.117209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=2451d53fb9974222827b6520b8bce5f3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:52.117711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.117878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.120549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.127760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 255/530] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/role_assignments?user.id=2451d53fb9974222827b6520b8bce5f3&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:52.128938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:52.139483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:52.140397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:52.141245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:52.148837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 276/531] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:52.148837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.307778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e80109-6ccc-4705-993b-4393d73c8b55 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.326206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.374462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.375071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.375348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.375873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.377345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:52.378286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.380933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.383442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa12b25-ae04-4e93-afd3-697c058c9c86 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.417647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 277/532] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:52.418547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.449069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.449689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.452092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.452386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.503727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.530877 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d773ba5-2624-42c8-b111-456ad9921622 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.533230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.534920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.566700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.567223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.567501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.567767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.568453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:52.568754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.570261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.582735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 278/533] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:52.583607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.615475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.647493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 256/534] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:52.648722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.700719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38960a28-eea3-4a81-8d44-dcc6c65e7a3f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.725901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.747885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.749492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.749970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.750975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.752591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=3a5e44f909a94c74990b2e80f1da5169) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:52.753101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.754349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d9679-a02f-4248-a7c3-0799d5074bf8 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.754994 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.762791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 279/535] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:52 2022] GET /identity/v3/role_assignments?user.id=3a5e44f909a94c74990b2e80f1da5169&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:52.763413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:52.816835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.817378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.817651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.818188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.818936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=e9754c63d24741be93896d6ef83be3a8) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:52.819244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.820733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:52.833164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 257/536] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:52 2022] GET /identity/v3/role_assignments?user.id=e9754c63d24741be93896d6ef83be3a8&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:52.918057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:52.927281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fbb248-5648-4436-9f19-cd0ba1f55109 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:52.977754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:52.978466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:52.979318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:52.979590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:52.996635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:52.998147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:53.016573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:53.017555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:53.019169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:53.019169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:53.019169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:53.019169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 280/537] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:53.025410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:53.151946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e9eddb-0b1e-4426-bf15-e85825eada48 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:53.192261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:53.192917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:53.193196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:53.193469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:53.194182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:53.194484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:53.195839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835cb2d8-432c-4246-a74b-294a2838dbff None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:53.207949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:53.216647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 258/538] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:53.222056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:53.356584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546efabd-e1df-458a-8146-319cb33129ba None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:53.369170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5619-ed80-4371-9ad9-3aef559828e4 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:53.369170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5619-ed80-4371-9ad9-3aef559828e4 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:53.369170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5619-ed80-4371-9ad9-3aef559828e4 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:53.371483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 259/539] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:53.376020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:53.376518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:53.376985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:53.395376 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:53.396897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:53.412984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:53.415177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:53.415694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:53.416194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:53.430630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:53.434175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:53.455594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:53.456475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:53.457304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:53.468434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 281/540] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:53.473825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:53.489687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:53.534022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 260/541] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:53.633911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bf1d5-1e03-421a-b524-89ed54f4aed8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:53.698781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:53.699637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:53.700060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:53.700469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:53.701388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:53.701832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:53.703501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:53.725298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 282/542] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:47:53.731390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:53.860115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:53.871717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570e9aac-5537-4494-b417-29dc7fd1ac75 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:53.922074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:53.937945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:53.939276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:53.940559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:53.941805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:53.942798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=266e3f374e794afca24caf914399a7d1) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:53.945930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:53.948140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:53.957807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 261/543] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:53 2022] GET /identity/v3/role_assignments?user.id=266e3f374e794afca24caf914399a7d1&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:53.999148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:54.003742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc622a84-d632-4e59-8739-5c6484f71e24 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:54.042850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:54.043910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:54.044454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:54.044959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:54.064268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:54.069810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:54.070789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:54.071584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:54.074919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 283/544] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:47:54.080892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:54.206264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064abf3-16bb-44a0-8e8e-a30b84a806fb None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:54.245622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:54.246633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:54.247143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:54.247611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:54.248488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:54.248983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:54.250830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:54.259754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 262/545] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:54.267092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:54.402243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271e939-b1e4-4ecb-a0e2-9c533cc55846 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:54.436670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:54.448852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:54.449486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:54.449763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:54.450049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:54.466085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:54.467552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:54.487540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:54.488516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:54.489980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:54.489980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:54.489980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:54.489980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 284/546] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:54.494301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:54.583710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:54.643434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b70cc-4008-48b4-b144-e9086d215683 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:54.697197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:54.698644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:54.699065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:54.699537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:54.700330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:54.700774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:54.706409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c086b5-a86b-4d5c-944e-2c63beb10aea None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:54.723404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 263/547] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:54.728493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:54.789067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d056a-9f10-4d98-8ef5-32fc88b47f05 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:54.789574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d056a-9f10-4d98-8ef5-32fc88b47f05 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:54.789762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d056a-9f10-4d98-8ef5-32fc88b47f05 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:54.792091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 264/548] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:54.799321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:54.799772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:54.800181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:54.821934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:54.823893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:54.910207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 265/549] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:54.959162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d8d8f-6841-4102-8410-40eddd4b9022 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.012530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.013161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.013438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.013710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.027047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:55.028500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:55.043465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:55.045141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:55.046528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 285/550] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:47:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:55.050487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.108219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:55.172278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3525aec-efb6-4c91-bd1f-bbcad5fb254a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.209354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.210230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.210656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.211126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.212049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:55.212490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:55.214613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7542fb-7554-44d9-bdcd-14868e29cb3f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:55.233317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 266/551] 10.222.0.50 () {56 vars in 1135 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:55.239055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.250803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342469-1317-4234-baad-54aee857bba3 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.251095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342469-1317-4234-baad-54aee857bba3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.251370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342469-1317-4234-baad-54aee857bba3 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.253293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 267/552] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:55.257708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.257973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.258281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.280568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:55.282274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:55.293292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:55.364637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3b54-73c8-4d13-b053-5b72450a91a5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.374030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 268/553] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:47:55.374030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.402236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:55.432799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.433870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.434318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.434738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.483983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aadb15-66dc-4c9f-9eb3-8c00f68db66c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.523473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.524101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.524381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.524653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.547585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:55.552406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:55.553482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:55.554406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 269/554] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:55.555121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:55.556042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:55.565502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.620268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:55.718499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 286/555] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:47:55.719390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.756443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:55.767644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c953b-856e-4ed7-9103-d186a6767633 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.811550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.812434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.812789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.813900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.814701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:55.815075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:55.816512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:55.827943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 270/556] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:55.829219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.849239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f0a73f-2119-4d73-a5ce-1b5910c1c7c3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.885374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.886004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.886290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.886559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.893346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:55.897905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:55.904765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:55.905855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:55.906902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:55.908182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 287/557] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:55.908793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:55.937422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047262d8-4fe4-4796-a9e4-a41bf3ef9365 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:55.971730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:55.972463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:55.972579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:55.973094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:55.985452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:55.986927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d4f40616-9fdd-4466-baca-6b3a016fe39c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:56.001646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:56.002805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:56.003950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:56.003950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:56.003950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:56.007789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 271/558] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:56.008357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d882a-24b8-4396-9933-fd78e88c1adc None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.008647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d882a-24b8-4396-9933-fd78e88c1adc None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.008911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d882a-24b8-4396-9933-fd78e88c1adc None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.010926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 272/559] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:56.011455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d882a-24b8-4396-9933-fd78e88c1adc None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.020498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.025214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222bfd-4caa-4125-ad87-24f9de2fd2ab None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.062059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.062812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.063090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.063361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.074434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.075856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:56.090246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:56.091324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:56.092218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:56.092218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:56.092218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:56.092218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 288/560] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:56.092218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.123846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d882a-24b8-4396-9933-fd78e88c1adc None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.144917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.159692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.160600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.161082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.161358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.162201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:56.162507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.163876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06253b-454e-46bc-a645-f3dbadc7b32e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.176815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 273/561] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:56.176815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.176815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.176815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.192039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:56.193224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:56.206302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cd1a5-6946-4d52-8d34-452e71922342 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.246218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.246744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.247020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.247286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.247922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:56.248499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.249694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.256765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.268005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 289/562] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:56.268005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.274135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 274/563] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:56.275025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.389047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3266c8-c375-447e-9eee-beb5979f1b14 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.398222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81fb48-737e-4fb4-93dc-f29a49ed1fc7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.399696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.426686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.427227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.427505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.427779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.428683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:56.428791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.430215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.436675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.437533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.438018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.438479 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.442673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 290/564] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:56.443617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.450077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.451767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:56.467463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:56.468449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:56.469317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:56.469317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:56.469317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:56.471570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 275/565] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:56.472815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.528486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.559135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747a85a-73b3-48c2-8cb4-88f1890490b6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.586292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f927573-2a4a-47d4-ba8f-51788e40f4f7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.595751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.596388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.596667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.597303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.604014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.605456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:56.616465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:56.617327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:56.618233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:56.621799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.622132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 291/566] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:56.622454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.622735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.623006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.623328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.630855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.632285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:56.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:56.644935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 276/567] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:56.645894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.659064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.727354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92add95f-5714-4984-9e07-b0338781b184 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.744548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bef03-a6bd-40bb-a8e2-ded06b523b2a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.754856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.762575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.763171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.763450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.763723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.764363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:56.764663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.766079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26545a-067d-4fe2-b419-256f42f4681e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.777028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 292/568] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:56.778212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf5131-8a97-4048-a5ba-a128facdf626 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.778552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf5131-8a97-4048-a5ba-a128facdf626 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.778744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.778867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf5131-8a97-4048-a5ba-a128facdf626 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.779326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.779606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.782148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.782272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 293/569] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:56.782272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf5131-8a97-4048-a5ba-a128facdf626 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.793695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:56.801053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:56.801967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:56.802803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:56.803647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 277/570] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:56.804050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.877639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:56.889151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf5131-8a97-4048-a5ba-a128facdf626 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.903751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365674f-2720-4273-8be8-7302753e7867 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:56.922854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.923452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.923728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.923997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.924672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:56.924969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.926449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.937164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:56.937754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.938265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.938367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 294/571] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:56.938531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.938829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:56.939027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:56.939325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:56.940729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92160d-c834-4c76-ba47-608dfcf7d941 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:56.958119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 278/572] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:56.959291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:56.959566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:56.959833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:56.977616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:56.979371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:57.001736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.043337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eed938-b1d4-4d2b-bf40-b9f958068438 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.045262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 279/573] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:47:57.046109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.077517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.078181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.078464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.078890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.089978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.091451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:57.103132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:57.104121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 295/574] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:57.105045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.118968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.156870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e92401-827d-4b3a-a7f6-3532f3d46644 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.190643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.191244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.191521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.191794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.192461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:57.192764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.194180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.202619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 280/575] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:57.203410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.217652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8085f-896c-4014-98fd-675247cad0d4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.225156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.256880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.257479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.258194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.258469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.259186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=b8a56d415755492fae33fc13f57c8f3c) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:57.259486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.261704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.267259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 296/576] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/role_assignments?user.id=b8a56d415755492fae33fc13f57c8f3c&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:57.268020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.321827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df8e0d-4a14-41b6-a235-6e0542f8159c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.360276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.362409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.363100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.363372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.363679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.377926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.379456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.393210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76efa6-fc25-42c2-afd5-25e6f1490646 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:57.395586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:57.396702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:57.397590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:57.397590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:57.397590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:57.397590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 281/577] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:57.397590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.439773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.440525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.440780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.441830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.441945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:57.442232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.443683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.485100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 297/578] 10.222.0.50 () {56 vars in 1137 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:57.485947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.495197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.527209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be104701-0c5d-4329-93e3-ba178dab58ea None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.562015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.562468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.562745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.563016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.573875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.575276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:57.589247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:57.590373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:57.591268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:57.591268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:57.591268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:57.591268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 282/579] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:57.591268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.598506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f8dba-9e03-4040-ade2-2a84f61bdee1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.630859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.637947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.638949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.639606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.639884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.656809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:47:57.661737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:57.662995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:57.663977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 298/580] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:57.667358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.701484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b1fd1-444a-40fe-a606-7ad3664422d2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.724774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.738293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.738898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.739175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.739448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.740093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:57.740391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.741768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.756023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 283/581] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:57.756789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.793423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098308dd-f29b-402d-9a6e-5e469f0650a8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.830308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.830882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.831143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.831426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.841902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.843409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:57.859848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:57.861237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:57.863766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:57.863766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:57.863766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:57.863766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 299/582] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:57.863766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.871558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa113e-ae3c-459e-af9d-512a6671621c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:57.906277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:57.906910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:57.907144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:57.907402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:57.908044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:57.908338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:57.909702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:57.915041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.925636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 284/583] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:57.925636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:57.974099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:57.982596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bb246-06a0-406a-9e01-97468e2af67d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.019358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.019703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.020051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.020787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:58.021164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.022579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.028205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb78a3f-c791-4ee9-87ba-5802cf8d487a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.031816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 300/584] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:58.036994 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.062300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.063002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.063358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.063698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.070556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.072070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:58.083558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:58.084467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 285/585] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:58.090757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.146554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5c717-307b-4e7e-9615-b53ef74376fc None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.154236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.185171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.185775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.186077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.186350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.186995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:58.187284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.188651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.196081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285356d2-5ebd-4b02-bf77-f4155d5fec6d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.198554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.204568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 301/586] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:58.204568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.239280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.240171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.241254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.241534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.250963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.253230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:58.268828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:58.269752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:58.270714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 286/587] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:58.271312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.311433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077dc04f-dd40-46b9-a99b-78cf699a231a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.347464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.348016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.348297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.348570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.359723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.361181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:58.376434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:58.377378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:58.383256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:58.383256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:58.383256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:58.383426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795b311-cb5c-478d-8df6-070acbe32bd2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.383585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 302/588] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:58.383585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.402437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.419877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.421229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.421514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.421788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.422513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:58.422824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.424292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.437668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 287/589] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:58.438578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.464587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.497713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58c56b-9615-4d7a-bd83-e611b85fe8de None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.534027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.534650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.534903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.535344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.541936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.543410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.551083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9538e3a-9a03-45f2-b147-9b72f2686a7d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:58.554538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:58.555390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:58.556297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 303/590] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:58.556800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.591627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.592484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.592906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.593458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.594492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:58.594802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.596223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.614023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 288/591] 10.222.0.50 () {56 vars in 1137 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:58.614627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.652643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.678294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19930f-2db1-4c05-bfe6-b4b6964ae6db None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.707216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.713978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.714846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.715701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.716151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.717204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:58.717676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.719241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.726961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd89fc-9dbc-4cb9-9f9c-0c4e1e4a2d70 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.736712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 304/592] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:58.736712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.765141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.765872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.766256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.766604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/2451d53fb9974222827b6520b8bce5f3/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.781651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=2451d53fb9974222827b6520b8bce5f3, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.783126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.798813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 289/593] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:58 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/2451d53fb9974222827b6520b8bce5f3/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:58.799945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.846699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef37dea-202c-46b3-a77a-d6e8458766b0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:58.892159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:58.892848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:58.893236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:58.894379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:58.895249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:58.895833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:58.897639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:58.907999 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:58.917281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 305/594] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:58.918831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:58.969282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b14492-7d45-4c5f-ad04-cbe13b74da20 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.012229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:59.019978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.020210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.020543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.020801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.032887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.034402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.049403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccf1b60-300f-46eb-b050-ccb1d3ec7fe3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:59.049610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:59.050724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:59.051793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:59.051793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:59.051793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:59.055299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 290/595] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:59.056125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.089920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.090849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.091266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.091762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.100898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.102571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:47:59.116773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:59.117928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 04:47:59.119038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:59.122297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 306/596] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:47:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:59.123235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.173141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546e133-a309-4e14-96bb-12452244b712 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.193551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:59.219707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.220954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.221983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.222898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/e9754c63d24741be93896d6ef83be3a8/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.241330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6cff7-14a0-48d1-8d74-dbc4acf9f25a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.245087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=e9754c63d24741be93896d6ef83be3a8, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.246987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.274260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 291/597] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:59 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/e9754c63d24741be93896d6ef83be3a8/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:47:59.274260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.295373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.298205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.298554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.298832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.299516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:59.299809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.303828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.330409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 307/598] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:59.337103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.342525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:59.481526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288369d7-a513-4053-afda-2e6ca3e94465 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.533368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45483639-efe4-4fb3-b1f0-0204e679db28 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.537684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.538750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.539268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.539750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.540724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:59.541282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.543235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.558373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:59.570079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 292/599] 10.222.0.50 () {56 vars in 1139 bytes} [Sat Sep 24 04:47:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:59.584172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.584367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.584856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.585139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.585429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.593727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.595202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:47:59.608117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:59.609077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:47:59.610084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:47:59.613735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 308/600] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:47:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:59.615079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.734969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea48545-b69f-4220-a1de-f6c6d85fb2c4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.763204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:59.771453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.772065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.772345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.772772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.773474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:47:59.773778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.775917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e7bd4-cddf-4778-962d-f73dee12447b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.778626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4ecd-40ad-41d6-a0b0-15633fec7ace None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.801467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 309/601] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:47:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:47:59.802890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175b37f-ac99-4dd9-9c6e-265c997ab81f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.803184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175b37f-ac99-4dd9-9c6e-265c997ab81f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.803455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175b37f-ac99-4dd9-9c6e-265c997ab81f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.805161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 310/602] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:47:59.805848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175b37f-ac99-4dd9-9c6e-265c997ab81f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:47:59.847341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.848547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.849149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.849703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.852028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:47:59.868063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.871027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-3ef78101-3b51-47fb-8541-c5545b59eb3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:47:59.902796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:47:59.904007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:47:59.905095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:47:59.905095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:47:59.905095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:47:59.911164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 293/603] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:47:59.911164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.911164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.911164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.930333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175b37f-ac99-4dd9-9c6e-265c997ab81f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:47:59.934082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:47:59.934737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:47:59.977879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:47:59.978937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:47:59.979370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:47:59.979789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/3a5e44f909a94c74990b2e80f1da5169/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:47:59.997881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=3a5e44f909a94c74990b2e80f1da5169, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:47:59.999645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:00.012841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 294/604] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:00.013811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:00.020339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 311/605] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:47:59 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/3a5e44f909a94c74990b2e80f1da5169/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:00.103976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:00.133548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c8556-eda3-4557-9b8a-232f09aad3fa None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:00.186393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:00.190673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:00.190673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:00.190673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:00.190673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:00.190673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:00.190673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:00.213938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 295/606] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:00.222235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:00.340648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:00.356081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652d401-29c3-4f40-9739-c28b4f797e46 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:00.392769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:00.393542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:00.393920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:00.394318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:00.401666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:00.403239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:00.414480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:00.415560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:00.417203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:00.418902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 312/607] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:00.424509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:00.450460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:00.615086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbdba1-2165-4998-90c1-35baed369e97 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:00.640056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4ddb0-f79d-458e-953f-8c4d87ac4bdf None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:00.695344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:00.696004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:00.696371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:00.696642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:00.697332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:00.698214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:00.699644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:00.701322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:00.703821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:00.704548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:00.704927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:00.705295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:00.714840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 296/608] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:00.716301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:00.720582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:00.722397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:00.723621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:00.724666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:00.726494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 313/609] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:00.735357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:00.750383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:00.890408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1fcc6-8de0-413d-9c08-d7323b1ca775 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:00.931390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fbbf2-8e00-4d76-bde3-dfa2d911b488 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:00.954679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:00.955884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:00.956332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:00.956610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:00.957280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:00.957620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:00.959025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:00.977466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 314/610] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:00.983485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:01.006151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:01.006839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.007143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.007452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/266e3f374e794afca24caf914399a7d1/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.034324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=266e3f374e794afca24caf914399a7d1, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:01.035775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:01.056945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 297/611] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:48:00 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/266e3f374e794afca24caf914399a7d1/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:01.086399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:01.105783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7ecf-5ece-4450-b4e1-e5402c8fa8b9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:01.143814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:01.144695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.145021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.145298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.158381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:01.159995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:01.180967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:01.182525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:01.183839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:01.183839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:01.183839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:01.183839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 315/612] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:01.190082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:01.254587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:01.311755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b26e4a-50b6-49b2-8b60-c4aef0bf3a6f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:01.350373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:01.351040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.351364 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.351639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.352280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:01.352575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:01.354578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:01.366772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 298/613] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:01.372895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:01.487261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d464b8-f894-4df1-bf93-9cedb5550ebb None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:01.516081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:01.530831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:01.531452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.531728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.531997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.542074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:01.543577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:01.559372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:01.560383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:01.561938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 316/614] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:01.566394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:01.675424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a63af9-664f-49fc-9332-c1e1bc2be6e3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:01.677880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:01.722883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:01.723514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.723789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.724057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.724876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:01.725175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:01.726814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936703f-014d-487f-80ec-c90a1a7a2de8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:01.747943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 299/615] 10.222.0.50 () {56 vars in 1135 bytes} [Sat Sep 24 04:48:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:01.753720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:01.837480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564300-ae50-4fd7-85da-865b5fb6f275 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.837827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564300-ae50-4fd7-85da-865b5fb6f275 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.838574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2564300-ae50-4fd7-85da-865b5fb6f275 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.840229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 300/616] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:01.854498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.854800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.855029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.881617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:01.882844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:01.906213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3dd35-46a3-44e3-ad36-bb3e2858dee9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:01.971642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:01.972408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:01.972660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:01.972936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:01.999649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:02.004125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:02.019868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 301/617] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-5f104d35-475c-4df2-914b-5b0f8200fb99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:02.026265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:02.027219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:02.028115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:02.028115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:02.028115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:02.028115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 317/618] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:02.033101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.104328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d0771-ad17-45c9-981f-c556dbeaa097 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.104610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d0771-ad17-45c9-981f-c556dbeaa097 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.104877 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8d0771-ad17-45c9-981f-c556dbeaa097 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.110095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 318/619] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:02.112170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.112506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.112787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.154106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:02.154338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:02.156311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:02.171790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb83542-76d2-4277-8d95-57998af2ac19 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.214718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:02.236806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.237423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.237696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.237964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.238643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:02.238936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:02.240292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:02.259383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 302/620] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:02.265276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.302079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 319/621] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:02.360812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.426271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2492555-c7cb-4e83-97fc-18ff5a2e6cf6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.490582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.491283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.491556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.491822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.504623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:02.506085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:02.524696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:02.526140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:02.527024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:02.528193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 303/622] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:02.537609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bcc0d-ea83-4348-a6a3-4726f8f1a0e1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.539288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.570386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:02.597156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.597843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.598147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.598418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.620663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:02.627539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:02.628688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:02.629894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:02.630721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 320/623] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:02.635873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.686474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:02.701031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6608c-ed2a-43ce-b21b-79fd935f17ac None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.736757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.737691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.738198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.738662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.739527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:02.739998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:02.741571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:02.745966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136f65c-6f6d-4542-97a3-2e39a165add8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.754876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 304/624] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:02.755946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.786396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.787039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.787318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.787587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.788456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:02.788665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:02.790122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:02.799557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 321/625] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:02.800456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.830705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:02.865049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54084131-9cf3-4962-91a4-f36802b6f46f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.892137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:02.901436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2c8b5-4c19-44ff-bbe9-c994da01b3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.902090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2c8b5-4c19-44ff-bbe9-c994da01b3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.902370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2c8b5-4c19-44ff-bbe9-c994da01b3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.902641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2c8b5-4c19-44ff-bbe9-c994da01b3f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.910122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddd6a8-da7c-42d0-80fa-cc8812932756 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:02.912900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2c8b5-4c19-44ff-bbe9-c994da01b3f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-bdb2c8b5-4c19-44ff-bbe9-c994da01b3f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:02.917829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:02.918917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:02.919940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 305/626] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:02.920794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd87f32-ac41-4c0b-903a-098936abb19b None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.920794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd87f32-ac41-4c0b-903a-098936abb19b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.921019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd87f32-ac41-4c0b-903a-098936abb19b None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.926575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 306/627] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:02.927356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd87f32-ac41-4c0b-903a-098936abb19b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:02.952895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:02.953938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:02.954443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:02.954924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/b8a56d415755492fae33fc13f57c8f3c/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:02.971091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=b8a56d415755492fae33fc13f57c8f3c, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:02.972716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:02.987027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 322/628] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:48:02 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/b8a56d415755492fae33fc13f57c8f3c/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:02.987968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.063043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd87f32-ac41-4c0b-903a-098936abb19b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.106450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:03.115310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.116147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.116525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.116897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.118077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdac51a-15d0-4911-a545-b7d0b43d634d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.131727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.134174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-cf581dbf-6efd-45e3-8371-70fa8fc78282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:03.157496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:03.161428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:03.162330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:03.162330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:03.162330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:03.163484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 307/629] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:03.164798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.165083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.165357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.167673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.168046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.168554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.168761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.169368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:03.169664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.171706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.174145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:03.184996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 323/630] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:03.184996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.193853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:03.195958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:03.286097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 308/631] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:03.286978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.298964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6646d-ea16-4ee6-8d82-c45a2aca67a3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.356187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.356723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.356995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.357263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.357904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:03.358224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.359585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.377217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 324/632] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:03.383895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.466871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043cb9e-6d23-4428-b6a2-ec3c0968e468 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.486477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:03.487385 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:03.525209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.526087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.526371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.526692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.547497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.548943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.556060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3d791-bb55-481f-801a-5763b706ed0f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:03.566190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:03.567145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:03.568012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:03.568012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:03.568012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:03.568012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 309/633] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:03.575671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.626932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.627554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.627938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.628221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.637077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.638727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:03.652040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:03.653024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:03.655900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 325/634] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:03.658718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.713435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146820cc-e0b0-4f09-b9fd-f9d5fea10d63 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.754076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.754683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.754967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.755236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.755883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:03.756177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.757536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.770104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 310/635] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:03.770742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.801061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c26a81c-4b0d-4b1d-a561-70704a6a6428 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.816801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:03.822393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:03.845305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.845940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.846244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.846764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.861227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:03.868310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:03.869231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:03.870167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 326/636] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:03.871389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.886097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073d06b-e0c4-4241-b635-58e59d242454 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:03.922604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:03.923208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.923482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.923756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.924462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:03.924765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:03.926220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9907c4e-5112-476a-94c5-4df2debe660c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:03.944045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 311/637] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:03.945246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b951458-3fe2-4e4b-abe3-7dbd93baa4a8 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:03.945554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b951458-3fe2-4e4b-abe3-7dbd93baa4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:03.945854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b951458-3fe2-4e4b-abe3-7dbd93baa4a8 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:03.950092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 312/638] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:03.950092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b951458-3fe2-4e4b-abe3-7dbd93baa4a8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:03.985619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938a81c-bd03-499a-be1a-704eae801c23 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.024466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.025106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.025389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.025770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.033892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.035418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.039277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:04.047920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:04.048874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 327/639] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:04.049822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.063394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b951458-3fe2-4e4b-abe3-7dbd93baa4a8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.092108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.098113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.098797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.099169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.099557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.100283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:04.100679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.102144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3b575-8bbd-462f-b173-acff376c8372 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.110437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 313/640] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:04.111568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.111845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.112110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.130756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:04.131922 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:04.162460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7310205-b5b1-4810-be66-93494bed7880 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.198853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.199487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.199768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.200043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.202130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 314/641] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:04.203573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.285616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.307424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.309139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.316338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a93f25-1143-48da-9ff5-6f4ee2d9b8b6 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.363580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.364586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.364694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.365155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.365624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.366708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:04.367201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.368942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.389589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 315/642] 10.222.0.50 () {56 vars in 1137 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:04.390898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.450180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 328/643] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:04.452271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.503450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c5d04-55ca-4036-bcfd-aead845a310d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.537976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.538627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.538906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.539176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.541043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.551040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.552578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:04.567518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:04.568697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:04.569770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:04.569770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:04.569770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:04.569770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 316/644] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:04.574522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.575765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a0433-46f6-4f4e-8a55-740c312363b5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.611327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.611934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.612209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.612479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.622233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.687945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.688369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccd446-b3aa-4984-aa0e-b4bccd0cf434 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.689407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.723955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.724661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.725000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.725332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.737435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.738967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:04.754704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:04.755817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:04.756850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:04.756850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:04.756850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:04.756850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 317/645] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:04.756850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.799790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 329/646] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:04.799790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.800060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.841488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:04.879357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a24f3-98a2-449a-8878-289d8a73b5cc None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.905324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faa419-bf98-4070-a49a-6da886075381 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:04.914909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.915539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.915812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.917026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.927483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:04.932708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:04.933788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 318/647] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:04.934811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:04.946420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:04.947027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:04.947313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:04.947640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:04.948278 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:04.948576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:04.949916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:04.966217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 330/648] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:04.966217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.039557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.045660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fab05-52bb-4b9d-a2b4-29adf15052a2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.063596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.067755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c7f46-e815-46e0-87cb-6edb6e6c1f9a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.081801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.082507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.082846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.083178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.083879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:05.084255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.085678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.101728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 319/649] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:05.101728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.107482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.108188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.108539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.108919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.120244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.122526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:05.139526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:05.140518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:05.141393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:05.141393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:05.141393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:05.145280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 331/650] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:05.146088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.210700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5afc0a-cb65-4ea2-8746-1172438cda36 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.246758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.247357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.247643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.247886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.248552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:05.248828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.250217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.258593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 320/651] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:05.261756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.264095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2586c2-017d-484e-8f1e-09c1c52efd71 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.283575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.303227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.303785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.304061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.304337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.311868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.313311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:05.325085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:05.326354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 332/652] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:05.327917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.341721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.363607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba81e0-272b-4d9c-8d1f-b2fd6de6f2c4 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.397172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.397807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.398109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.398383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.404834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.406335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:05.417372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:05.418259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:05.419140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 321/653] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:05.419615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.429908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286265e3-9ffc-4472-9efd-62661577524d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.463696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.464289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.464564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.464830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.465465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:05.465758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.467145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebeee8-3dae-4c8d-849f-a097a6342ea4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.478317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 333/654] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:05.479488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8c113-cf71-46eb-a5fd-390dd95ca787 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.479826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8c113-cf71-46eb-a5fd-390dd95ca787 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.480025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8c113-cf71-46eb-a5fd-390dd95ca787 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.481335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 334/655] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:05.481855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8c113-cf71-46eb-a5fd-390dd95ca787 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.503455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.522234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c192b-5a21-474b-8dfc-49346b24fe7b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.550422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.554126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.554686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.554957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.555223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.566479 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.567899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:05.581455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:05.582472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:05.583344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:05.583344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:05.583344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:05.583520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8c113-cf71-46eb-a5fd-390dd95ca787 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.583591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 322/656] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:05.583591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.617702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.618349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.618616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.618890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.620150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:05.620452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.621872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.638957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 335/657] 10.222.0.50 () {56 vars in 1137 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:05.639816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.682866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445d7ed-9f29-47ec-b320-32aaceffbb23 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.711062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.716519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.717116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.717388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.717657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.718373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:05.718671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.720084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c493c-3a1f-4597-be76-199d98e75cd2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.732034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 323/658] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:05.733148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.733427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.733695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.739014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc14e14-66b2-4110-b6a0-6217a7370094 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.750630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:05.752064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:05.766136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.774643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.775269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.775576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.776099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.782632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.784077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:05.795861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:05.796760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 336/659] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:05.797659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.834092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 324/660] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:05.834092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.898095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cd827-f334-4d14-9e51-b3d40289818d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.926443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968c3f44-d158-4c56-936e-f7f0867ca2b6 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:05.931303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.932523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.932964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.933241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:05.933880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:05.934223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:05.935595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:05.950044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 337/661] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:05.950044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:05.954773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:05.967580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:05.968191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:05.968495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:05.968738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.000059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.037120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.038633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.057128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44c28e-0f96-4a6c-8aa8-c7ccc5a5a143 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.094608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.095232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.095412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.095653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.096366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=2451d53fb9974222827b6520b8bce5f3) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:06.096668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.098135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.103677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 338/662] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/role_assignments?user.id=2451d53fb9974222827b6520b8bce5f3&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:06.105001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.167128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 325/663] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:06.167128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.192921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.218615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3741f-8047-4125-b61b-7d0e05a70911 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.233107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.258560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.259324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.259433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.259705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.261027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:06.261253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.262731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.280401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 339/664] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:06.280401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.280683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a613f-824f-4d40-8cbf-deef0dc43588 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.316286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.316923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.317202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.317565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.325637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.327498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:06.342119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:06.343095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 326/665] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:06.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.395427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b34991-ff59-40f1-91dd-c29d3af6c5bd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.430269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.430900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.431171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.431441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.432066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.443451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:06.449045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:06.449970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:06.450832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:06.451900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868f5b1-41bd-4c93-8596-049d54c3f836 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.454459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 340/666] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:06.454954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.487149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.487775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.488050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.488317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.490419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.498973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.500410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:06.515180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:06.516120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:06.517038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:06.517038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:06.517038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:06.517038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 327/667] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:06.517038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.571112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f616e6-e3a7-4666-9576-af597c68cf6d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.608400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.609001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.609554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.610416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=e9754c63d24741be93896d6ef83be3a8) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:06.610722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.612279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.613704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eb34a-7458-4e12-820c-6f6e43cc2847 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.630033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 341/668] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/role_assignments?user.id=e9754c63d24741be93896d6ef83be3a8&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:06.630033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.666023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.667282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.667768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.668192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.668945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:06.669247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.671501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.673594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.697133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 328/669] 10.222.0.50 () {56 vars in 1139 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:06.698551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.755214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:06.766902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89d5cc-cb13-4184-ae47-2c01affdba22 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.806379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.807149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.807496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.807833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.808566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:06.808928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.810453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.820791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 342/670] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:06.821647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.836598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378ab14-e5a7-40c0-95fe-c92002a4dd54 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.885122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.885862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.886169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.886697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:06.887640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:06.887949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:06.889334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:06.907089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 329/671] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:06.907684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:06.950054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e017cb-65ce-4d04-87aa-a9805c56db44 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:06.997014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:06.998046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:06.998947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:06.999428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.001555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.029445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7021f4c-5921-4d37-a2bb-4576b3c89881 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.039529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.076545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.078213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.078988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.079486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.095325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.096785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.097066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.099019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:07.120154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:07.121210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:07.122124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:07.122124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:07.122124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:07.126510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 330/672] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:07.126510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 343/673] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:07.237105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.244975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb0ab0-4dc8-4cb6-8d95-ee10781fb9b6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.248746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.287148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.287774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.288164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.288501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.295893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.297409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.308647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:07.309496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:07.310575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:07.311558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 331/674] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:07.312059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.373628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702510d7-514a-4ca3-bf66-45a2a5dee033 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.413003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.413617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.413892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.418018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.418018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:07.418018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.418018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.426407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f4aca-3648-4441-9451-1c2ff5bb9d1b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.436267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 344/675] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:07.437302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.471452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.472208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.472487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.472761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.484998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.486828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:07.502832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:07.505006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:07.507089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:07.507089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:07.507089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:07.514017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 332/676] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:07.514017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.529756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.553564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ef441-f02b-425e-ad48-1181d714c0c1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.572111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.594667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.595276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.595555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.595824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.597012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:07.597318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.599243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.614300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 345/677] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:07.615247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.624046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6bb58f-776b-4f3a-bccb-3229c0f86872 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.668698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.669449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.669777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.670086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.679324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.680864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:07.694832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:07.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 333/678] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:07.699277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.738592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e40ace-ffdb-4a5b-b766-4a8ac12e78ab None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.779700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.780308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.780584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.780853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.781488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:07.781781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.783171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.784644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.797815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 346/679] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:07.798800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.811148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c15af-7b5f-4bbc-8b93-09cfae57baea None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.846234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.846536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:07.847258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.847544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.847821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.848764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=3a5e44f909a94c74990b2e80f1da5169) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:07.849064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.850940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.857597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 334/680] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/role_assignments?user.id=3a5e44f909a94c74990b2e80f1da5169&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:07.858519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:07.925034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02eb01e-8073-4b6a-9473-dfc2ca1d6120 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.968915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:07.969632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:07.970019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:07.970296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:07.971032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:07.971332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:07.973433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:07.976454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8951bc-6a79-4737-af4d-e9ab373babf7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:07.995860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 347/681] 10.222.0.50 () {56 vars in 1135 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:08.003166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:08.044403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:08.045055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.045338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.045590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.063995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:08.065485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:08.089290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:08.093653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:08.095687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:08.097760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 335/682] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:08.102019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:08.176809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:08.183644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3fb22-0973-4ad0-93c7-e48367ab69ad None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:08.219996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:08.220775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.221045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.221381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.242422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927cc8d-71db-4714-9086-179e8fc6f675 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:08.282097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:08.282981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.283434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.283897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.284845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:08.285377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:08.287160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:08.294928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:08.296426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:08.302192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 336/683] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:08.308722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:08.410423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:08.448861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:08.462845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 348/684] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:08.464779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406c503-3c2c-4974-8c9a-c0253d740dcc None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:08.466853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:08.523249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:08.524229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.524691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.525158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.526628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=266e3f374e794afca24caf914399a7d1) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:08.527662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:08.529738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b7d1a-6055-4521-b9ca-d13a174e9769 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:08.539071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 337/685] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:08 2022] GET /identity/v3/role_assignments?user.id=266e3f374e794afca24caf914399a7d1&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:08.540622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48c8c3-6a8c-4d59-8610-5a18396a446c None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.541092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48c8c3-6a8c-4d59-8610-5a18396a446c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.541603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48c8c3-6a8c-4d59-8610-5a18396a446c None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.544017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 338/686] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:08.551752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.551945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.552050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.592240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:08.593647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:08.648327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6a82f-616e-4f52-b46f-a2187aa68ee3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:08.703011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:08.703660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.703940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.704212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.715996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:08.717531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:08.733661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:08.734705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:08.735579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:08.735579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:08.735579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:08.735579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 349/687] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:08.741831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:08.743573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 339/688] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:08.802462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:08.848599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7df23-168c-423e-990d-bf0f8030e4f4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:08.883329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:08.883937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:08.884213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:08.884485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:08.885645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:08.885946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:08.887356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:08.900087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 350/689] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:08.905548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:08.970468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:09.022058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd890c-3f7a-4c80-b5fa-19a7f024e6ed None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.057094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.057651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.057950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.058257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.062550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.066748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:09.068547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:09.083860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:09.084971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:09.086093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:09.097636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 340/690] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:09.099610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.217270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0e290-c40c-4ada-9d08-158cb703f8da None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.222502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d32ba9-f36b-4bee-b786-43ee6cea8032 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.228372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:09.239861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:09.260368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a785db-f6bd-4786-b290-3c305fc0bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.261012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a785db-f6bd-4786-b290-3c305fc0bec7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.261294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a785db-f6bd-4786-b290-3c305fc0bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.261736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a785db-f6bd-4786-b290-3c305fc0bec7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.264589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.265361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.265780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.266217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.267195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:09.267639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:09.269688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da3299-b177-48ef-9029-9568656f9629 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:09.272261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a785db-f6bd-4786-b290-3c305fc0bec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-f1a785db-f6bd-4786-b290-3c305fc0bec7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:09.278063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:09.279322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:09.280279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 351/691] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:09.280968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68ea6e-bd79-489b-b99e-c231f4d1f35b None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.280968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68ea6e-bd79-489b-b99e-c231f4d1f35b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.285699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68ea6e-bd79-489b-b99e-c231f4d1f35b None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.286138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 352/692] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:09.287586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68ea6e-bd79-489b-b99e-c231f4d1f35b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.299092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 341/693] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:09.300802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.301265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.301816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.321731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:09.327936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:09.408713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68ea6e-bd79-489b-b99e-c231f4d1f35b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.413075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 342/694] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:09.414160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.469056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.469662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.469937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.470229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.470885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:09.471179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:09.472533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:09.494936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 353/695] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:09.499778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.558779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:09.598889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d817b1-6958-4332-a8c5-64372db32e36 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.604396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:09.655296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c96e7-eff9-4f25-8db0-b13f865e4b12 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.672352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.672962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.673237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.673504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.674352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=b8a56d415755492fae33fc13f57c8f3c) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:09.674661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:09.676079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:09.688737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 343/696] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/role_assignments?user.id=b8a56d415755492fae33fc13f57c8f3c&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:09.714708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.715757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.716235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.716723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.728126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.729816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:09.731363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:09.746558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:09.747600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:09.748478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:09.748478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:09.748478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:09.748478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 354/697] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:09.753912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.843880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:09.863946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec9991-5a37-460c-97b0-9d7545a10568 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:09.910751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:09.911379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:09.911655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:09.911928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:09.912579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:09.912881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:09.914324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:09.938640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 355/698] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:09.946218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:09.967030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9cc408-dcdb-48f4-bff1-7b1b9eda54a7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.015156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.016366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.016946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.017506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.035574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.037529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:10.057280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:10.058267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:10.059141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:10.059141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:10.059141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:10.059141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 344/699] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:10.071000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:10.102635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf88a2-3269-49ca-a28a-57e21e0252ce None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.103511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:10.146608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.147160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.147492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.147765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.158795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.160756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:10.176347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:10.177359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:10.178814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 356/700] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:48:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:10.184372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:10.187707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:10.208043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4d445-4846-4b98-a91d-2c6c2de3a031 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.247605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.248285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.248620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.248809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.249461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:10.249756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.251436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.268411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 345/701] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:10.270700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:10.300306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6ff64-f5b8-4446-9163-3782dec03366 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.352200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.353331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.354404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.354948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.355953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:10.356524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.358347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.359044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:10.381043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97510b69-5471-4403-a7cf-68981146b5e4 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.382400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 357/702] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:48:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:10.391346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:10.426875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ed8ed-842d-4e97-9a9a-0b0a0e8ccfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.427408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed8ed-842d-4e97-9a9a-0b0a0e8ccfa8 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.427789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed8ed-842d-4e97-9a9a-0b0a0e8ccfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.427958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ed8ed-842d-4e97-9a9a-0b0a0e8ccfa8 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.428602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed8ed-842d-4e97-9a9a-0b0a0e8ccfa8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.430079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ed8ed-842d-4e97-9a9a-0b0a0e8ccfa8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.471088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:10.474066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 346/703] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:48:10 2022] POST /identity/v3/users => generated 254 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:10.518442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba64672-d1de-46d6-a279-57031f8d117b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.556058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.557151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.557624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.558131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.598162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a446c-137b-4f32-be24-4f8313da47fe None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.598162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a446c-137b-4f32-be24-4f8313da47fe None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.598471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a446c-137b-4f32-be24-4f8313da47fe None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.599729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 347/704] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:10.611830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.612172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.612379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.636626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.638530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.662687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:10.663537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:10.742933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:10.752326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 358/705] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:10.758101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:10.815555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 348/706] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:10.876973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9aa51-6df9-40cc-93b3-ebc2dd335690 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:10.913131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:10.914054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:10.914472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:10.914878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:10.926055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:10.927780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:10.943937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:10.945101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:10.946215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:10.946215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:10.946215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:10.948046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 359/707] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:10.955075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.061779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:11.112164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0c1a3-bf91-404e-b88e-c64599e0e2c7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.133375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.151405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:11.157755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.158739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.159092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.159433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.160157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:11.160528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.166034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.188848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 349/708] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:11.207923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.304772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5bf3f-643d-4092-b47b-5ca829ba1f88 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.356749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.357690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.358213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.358751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.359629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.361166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.368213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac5b32-4d05-4b81-9e0c-c59ac7a50a20 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:11.382835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:11.383954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:11.384807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:11.387745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 360/709] 10.222.0.50 () {56 vars in 1129 bytes} [Sat Sep 24 04:48:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:11.404640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.415529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:11.436433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.437068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.437351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.437622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.453001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.454755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-fb9e2e96-ce97-4e92-bb0e-bad1b0cd6ef7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:11.468131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:11.469038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:11.469921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:11.474116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 350/710] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:11.475582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dc0ce-6afe-4d5a-b977-fadd16c84834 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.475871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dc0ce-6afe-4d5a-b977-fadd16c84834 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.476142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1dc0ce-6afe-4d5a-b977-fadd16c84834 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.477636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 351/711] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:11.482210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dc0ce-6afe-4d5a-b977-fadd16c84834 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.495505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:11.522570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0eb36-0b44-4da6-9b42-8e842abb44d9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.562392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.563989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.564395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.564770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.565510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:11.565985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.567491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba062c-1022-43ca-8ded-0fbc97231e20 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.573661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 361/712] 10.222.0.50 () {56 vars in 1124 bytes} [Sat Sep 24 04:48:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:11.575458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.576118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.576491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.597962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:11.599338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:11.600691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1dc0ce-6afe-4d5a-b977-fadd16c84834 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.655808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.656927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.657279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.657906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.658706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:11.659083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.661291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.678886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 352/713] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:11.680206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.685524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 362/714] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:11.686559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:11.739712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:11.791877 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:11.825211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa4852-aa71-4894-88ea-ad34381db607 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.864308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.864922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.865197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.865470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.866236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=2451d53fb9974222827b6520b8bce5f3) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:11.866633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.868089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.874348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 363/715] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:11 2022] GET /identity/v3/role_assignments?user.id=2451d53fb9974222827b6520b8bce5f3&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:11.878164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35bbae-7566-4761-a883-0c71984fd2b3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:11.959402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:11.960035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:11.960319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:11.960585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:11.961239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:11.961532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:11.963025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:11.975847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 353/716] 10.222.0.50 () {56 vars in 1124 bytes} [Sat Sep 24 04:48:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:12.014865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:12.157607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a578de5-2eaf-4921-8fe8-413879332ea7 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:12.196392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:12.197034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:12.197314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:12.197618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] PATH_INFO: `/v3/services/network` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:12.198265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:12.199634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:12.201460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:12.210055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:12.211075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:12.211892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 364/717] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:12 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:12.219459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:12.329102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c4b43-d641-43d5-a0b5-515e281da56a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:12.341856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:12.374346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:12.375806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:12.376550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:12.377019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:12.377867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] RBAC: Adding query filter params (name=network) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:12.378422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:12.380035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b26a88-c506-4d71-8052-39d8d286b135 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:12.386395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 354/718] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:12.392197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:12.521534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce45111-ed57-4e4d-9366-9fc350858c9c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:12.549506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb8dd3-d77b-4fcf-9c3c-3f23a9869132 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:12.550175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb8dd3-d77b-4fcf-9c3c-3f23a9869132 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:12.550655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb8dd3-d77b-4fcf-9c3c-3f23a9869132 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:12.552184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 355/719] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:12.564171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:12.564747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:12.565368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:12.566487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:12.567092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:12.567367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:12.567636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:12.568261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] RBAC: Adding query filter params (type=network) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:12.568556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:12.574025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:12.576764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 365/720] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:12 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:12.594450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:12.595966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:12.707853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 356/721] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:48:13.107815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:13.181456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:13.229953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de75a392-47d2-475a-b405-7611cb26d548 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:13.278611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9aec1-8592-4f2c-b4f2-87d4468aadd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:13.279245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9aec1-8592-4f2c-b4f2-87d4468aadd4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:13.279521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9aec1-8592-4f2c-b4f2-87d4468aadd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:13.279790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9aec1-8592-4f2c-b4f2-87d4468aadd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:13.301287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9aec1-8592-4f2c-b4f2-87d4468aadd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-bfd9aec1-8592-4f2c-b4f2-87d4468aadd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:13.306685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:13.307709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:13.309207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 366/722] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:13.314044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:13.446139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe1c6c-7211-4407-8ab1-78ff9055d0f5 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:13.451012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:13.452611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a5957-1716-4d05-90ec-a5f555478f0c None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:13.453380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a5957-1716-4d05-90ec-a5f555478f0c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:13.453822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a5957-1716-4d05-90ec-a5f555478f0c None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:13.455330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 367/723] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:13.463925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:13.464208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:13.464486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:13.486610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:13.487804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:13.512577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:13.513193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:13.513467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:13.513733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:13.514436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:13.514818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:13.516220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:13.535442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 357/724] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:13.541261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:13.600267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 368/725] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 24 04:48:13.716439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d0d8f-6028-48d1-aff0-bca4ca452d2f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:13.768002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:13.768989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:13.769465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:13.770054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:13.783946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:13.785607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:13.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:13.809595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:13.810547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:13.810547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:13.810547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:13.812323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 358/726] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:13.818206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:13.883128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:13.945729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224b75c-a123-4868-adaa-4e5419fde10e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:13.995846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:13.996375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:13.996653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:13.996925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:13.997568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:13.997862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:13.999254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:14.017065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 369/727] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:14.022169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.048487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:14.131803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.159526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45383b6a-4f98-4518-9ed0-128eedf057df None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:14.205729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:14.206399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.206675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.206947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.221244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:14.222735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:14.244739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:14.245728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:14.246748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:14.258438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 359/728] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:14.259367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.332206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33de02-60e1-4e43-94d6-4ef2e0c2e861 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:14.362351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:14.402769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14691336-06d3-482e-b204-be5b95193dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:14.403291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:14.404370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14691336-06d3-482e-b204-be5b95193dec None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.404645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14691336-06d3-482e-b204-be5b95193dec None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.404920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14691336-06d3-482e-b204-be5b95193dec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.430337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14691336-06d3-482e-b204-be5b95193dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-14691336-06d3-482e-b204-be5b95193dec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:14.435935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:14.436943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:14.437791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 370/729] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:14.447161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14691336-06d3-482e-b204-be5b95193dec None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.464367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d320a-a12b-4bf6-9800-142389ef4052 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:14.537514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:14.538548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.541699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.543895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.545937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:14.547394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:14.550747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7718e0dc-2cd4-474b-821f-3a4f20785fcc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:14.573784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 360/730] 10.222.0.50 () {56 vars in 1135 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:14.575523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361720af-a6db-4efe-b6c9-9567c80a8466 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.576058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361720af-a6db-4efe-b6c9-9567c80a8466 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.576625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361720af-a6db-4efe-b6c9-9567c80a8466 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.578403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 361/731] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:14.579301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361720af-a6db-4efe-b6c9-9567c80a8466 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.618184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14691336-06d3-482e-b204-be5b95193dec None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:14.663441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:14.664046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.664381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.664653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.665313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:14.665609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:14.667044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b00916-720d-4d66-a787-b73f292971ac None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:14.678055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 371/732] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:14.678055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedb194-4f84-4375-bbfe-ebebbb911f76 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.678055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedb194-4f84-4375-bbfe-ebebbb911f76 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.678055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedb194-4f84-4375-bbfe-ebebbb911f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.695286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1fedb194-4f84-4375-bbfe-ebebbb911f76 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:14.696461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1fedb194-4f84-4375-bbfe-ebebbb911f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:14.700394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361720af-a6db-4efe-b6c9-9567c80a8466 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:14.724440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:14.735835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9530a-9d68-44d5-b975-e299b09dab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:14.736434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9530a-9d68-44d5-b975-e299b09dab32 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.736707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9530a-9d68-44d5-b975-e299b09dab32 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.736973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9530a-9d68-44d5-b975-e299b09dab32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.741164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:14.767936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 372/733] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:14.769910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c9244-a756-49d0-93e9-d2443058a0ec None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.770431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c9244-a756-49d0-93e9-d2443058a0ec None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.770927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c9244-a756-49d0-93e9-d2443058a0ec None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.776589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 373/734] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:14.776589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c9244-a756-49d0-93e9-d2443058a0ec None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.820119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9530a-9d68-44d5-b975-e299b09dab32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:14.821514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9530a-9d68-44d5-b975-e299b09dab32 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:14.903826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c9244-a756-49d0-93e9-d2443058a0ec None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:14.931692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 362/735] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:14.933362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.933849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.934382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.942576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:14.943200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:14.943483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:14.943746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:14.954233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:14.954768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:14.955963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:14.956193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:14.974035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:14.975102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:14.976009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:14.976009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:14.976009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:14.976009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 374/736] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:14.976009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:14.994750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:14.998041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.037661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 363/737] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:15.038695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.109610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b191fa-4263-4ee0-967c-a1fa4a83b203 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.150447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.151653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.151928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.152236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.164236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.164690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e56a44-0270-4aed-955c-12ba27f7990c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.165694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:15.183941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:15.184907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:15.185780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:15.185780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:15.185780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:15.185780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 375/738] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:15.185780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.209766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.211087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.211800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.212244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.213719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:15.214397 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.215772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.231861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 364/739] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:15.231861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.259553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.263844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.302430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5bb590-02e6-4af1-b507-f982fbd63826 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.335060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5855a-4d79-4a03-88f4-bea4ab457111 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.336028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bb53b-9829-47df-a106-5d7f7751ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.336649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bb53b-9829-47df-a106-5d7f7751ea00 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.336925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bb53b-9829-47df-a106-5d7f7751ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.337194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bb53b-9829-47df-a106-5d7f7751ea00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.347644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bb53b-9829-47df-a106-5d7f7751ea00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-602bb53b-9829-47df-a106-5d7f7751ea00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:15.352765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:15.353683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:15.355209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 376/740] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:15.355896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035e46-37b6-4887-a2c4-6bbe82eb240a None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.355896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035e46-37b6-4887-a2c4-6bbe82eb240a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.356079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035e46-37b6-4887-a2c4-6bbe82eb240a None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.357435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 377/741] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:15.357950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84035e46-37b6-4887-a2c4-6bbe82eb240a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.371563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.372108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.372383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.372652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.373292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:15.373590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.374969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.386701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 365/742] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:15.387462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.464680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84035e46-37b6-4887-a2c4-6bbe82eb240a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.470711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.486348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efa696-9efd-4814-8a40-8e9ad7c838af None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.491468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.499673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.500307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.500584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.500868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.508998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.511003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.520276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6436c-0564-43b0-ad40-155034a78a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.520870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6436c-0564-43b0-ad40-155034a78a2d None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.521143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6436c-0564-43b0-ad40-155034a78a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.521411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6436c-0564-43b0-ad40-155034a78a2d None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.522084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6436c-0564-43b0-ad40-155034a78a2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:15.522377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:15.523437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:15.524574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:15.525337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6436c-0564-43b0-ad40-155034a78a2d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.525435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 378/743] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:15.525435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.544833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 366/744] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:15 2022] POST /identity/v3/services => generated 251 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:15.546341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.546632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.546909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.567308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:15.568559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:15.644279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef806-a9a9-4206-8566-fab37f854af0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.656558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 367/745] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 24 04:48:15.657406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.696742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.698387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.698894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.699389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.700330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:15.700833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.703066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.716254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 379/746] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:15.717724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.736976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.785036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3052b0-627d-4b6a-97de-b379fa510c4d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.796662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.820559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.821198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.821528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.821806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.829297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.830838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:15.843222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:15.845842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 368/747] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:15.848299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.858371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233202-1681-4f14-9478-0eeda8a2d87f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.900243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.900813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.901196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.901530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:15.902411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:15.902764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:15.904273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:15.926028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 380/748] 10.222.0.50 () {56 vars in 1149 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:15.926883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:15.955252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6db4ee-a10a-4fd5-8710-b4efcab17b0a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:15.987205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:15.992681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:15.993315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:15.993594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:15.993854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.005379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.006869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:16.021792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:16.022966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:16.024063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:16.024063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:16.024063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:16.024705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 369/749] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:16.025413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.041194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8616-e31e-4871-8c91-bdb464a5f742 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.052516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.083992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.084980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.085353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.085629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.086367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:16.086667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.088767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.106951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 381/750] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:16.107782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.138077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9faf2-ad28-4c46-9b1e-e461815de3a8 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.172363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.172875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.173149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.173421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.225081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626f998-ab90-432e-9605-ba0c528c323b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.226395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.242517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.244077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.265119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.265729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.266018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.266309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.266954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.268315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.283903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 382/751] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:16 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:16.285015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.329489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.393571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 370/752] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 24 04:48:16.393571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.445257 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d6b0c-46a1-4d41-985c-00b0b6e48813 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.484955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.485570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.485847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.486168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.486815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:16.487114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.489429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.504097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 383/753] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:16.505067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.515622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.541072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3baf0a-f706-488c-93a0-8f7dfc0e2a92 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.580934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.581536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.581810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.582122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.582842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=b8a56d415755492fae33fc13f57c8f3c) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:16.583144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.584545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.590891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 371/754] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/role_assignments?user.id=b8a56d415755492fae33fc13f57c8f3c&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:16.591769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.608418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.633117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8bf10-b2af-4786-a27c-3ffa20bf40ba None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.686665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.688143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.689307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.689812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.704786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.706621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:16.725951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:16.727016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:16.727896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:16.727896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:16.727896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:16.731331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 384/755] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:16.737716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.757864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec230ec8-1335-4be1-83a4-fc6c29abd6f1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.832937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.833584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.833858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.834156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.847157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.848939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:16.866049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:16.867035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:16.868054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:16.872651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 372/756] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:16.873520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:16.886632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.906671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2c04e-d31f-4096-bc0f-71cf45c2d0a9 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:16.961859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:16.962504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:16.962788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:16.963059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:16.964249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:16.964546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:16.965914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:16.973822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:16.992791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 385/757] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:16.999698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:17.044175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf74684-fef3-4e21-b050-e8b92b7bdf2d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.098481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.100313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.100789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.101411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.103078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:17.104227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.107175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.141382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 373/758] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:17.145519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:17.181990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd152da-2e71-4132-bc44-57839d8922e1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.237817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.238980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.239454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.239916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.247707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.249381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:17.263903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:17.264801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:17.265706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:17.268767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 386/759] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:17.274713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:17.294911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:17.356366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212bc759-db81-4e4b-b5e1-77c8e5b11d39 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.394984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.396042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.396513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.397030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.401824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:17.409215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.410796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:17.426702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:17.427656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:17.428534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:17.428534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:17.428534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:17.428534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 374/760] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:17.436237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:17.522371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9568d-4bc4-4b0e-b3e5-b3b452e92239 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.539602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cacdc3e-ef99-46db-b236-8f6681268ffa None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.563029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:17.585715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.586347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.586621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.586887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.587572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:17.587869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.589863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.602658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.603268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.603543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.603963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.605115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:17.605411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.606785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.622358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 387/761] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:17.628588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:17.635784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 375/762] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:17.643478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:17.819093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae0c4c6-daf3-4ed0-b12f-f1ea31785126 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.840233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:17.844764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d4ffb-1840-4f47-8f29-09f7fc7a5d53 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:17.869146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.870401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.870657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.870924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.871671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=8b1bc0d6bb8248cd86c8ac462709e326) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:17.871969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.873441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.887968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 388/763] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:17 2022] GET /identity/v3/role_assignments?user.id=8b1bc0d6bb8248cd86c8ac462709e326&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:17.909899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:17.910630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:17.910917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:17.911185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:17.922783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:17.924250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:17.936822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:17.938959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:17.941088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:17.946017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 376/764] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:17.948420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:18.187917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4674437-8056-46e7-a0c2-209d21cc1266 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:18.225844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:18.226480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:18.226798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:18.227074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:18.227752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:18.228055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:18.230077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ae51f-3bcf-4b87-9c7a-79bb70e16b04 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:18.242385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 389/765] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:18.251085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:18.272181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da146fc-0995-48ee-a346-b7efa365522e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:18.272490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da146fc-0995-48ee-a346-b7efa365522e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:18.272902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da146fc-0995-48ee-a346-b7efa365522e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:18.274925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 390/766] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:18.282526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21f0c-5383-4997-a566-f05addc964d7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:18.282812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21f0c-5383-4997-a566-f05addc964d7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:18.283088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21f0c-5383-4997-a566-f05addc964d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:18.318536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:18.319386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a9c21f0c-5383-4997-a566-f05addc964d7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:18.320601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a9c21f0c-5383-4997-a566-f05addc964d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:18.383109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:18.412150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c41052b-7531-4afe-bbf7-5f87e3c7ac82 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:18.447282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 391/767] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:18.472725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17726579-e988-4135-8364-8c55a8687683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:18.473402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726579-e988-4135-8364-8c55a8687683 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:18.473675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726579-e988-4135-8364-8c55a8687683 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:18.473948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726579-e988-4135-8364-8c55a8687683 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/2451d53fb9974222827b6520b8bce5f3/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:18.476154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17726579-e988-4135-8364-8c55a8687683 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=2451d53fb9974222827b6520b8bce5f3, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:18.477618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17726579-e988-4135-8364-8c55a8687683 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:18.508017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 377/768] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:48:18 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/2451d53fb9974222827b6520b8bce5f3/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:18.773247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe95cb9-0fb2-45e9-ac59-653fc7a31904 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:18.773580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe95cb9-0fb2-45e9-ac59-653fc7a31904 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:18.773789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe95cb9-0fb2-45e9-ac59-653fc7a31904 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:18.775136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 392/769] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:18.782221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:18.783049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:18.783497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:18.803968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:18.805453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:18.853326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe95cb9-0fb2-45e9-ac59-653fc7a31904 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:18.887856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 378/770] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:18.961672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe95cb9-0fb2-45e9-ac59-653fc7a31904 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.000579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:19.001839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.002352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.002818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.003615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:19.005182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:19.014793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:19.015777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:19.016617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:19.019303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 393/771] 10.222.0.50 () {56 vars in 1129 bytes} [Sat Sep 24 04:48:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:19.027372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.056722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:19.152286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb0ecd-abba-4ddf-a791-67a2693eb9bf None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.195224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:19.195836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.196113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.196383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.197021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:19.197316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:19.198823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:19.206021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 379/772] 10.222.0.50 () {56 vars in 1124 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:19.211510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.252459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:19.323715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c682e35-93c0-4787-88e2-7d8393f95347 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.339467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.374522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:19.375140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.375419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.375688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.376326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:19.376879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:19.378282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:19.395612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 394/773] 10.222.0.50 () {56 vars in 1124 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:19.398568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.503424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:19.514068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8faa03-00b2-4496-987f-ba086c659914 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.559221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:19.560233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.560907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.561322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] PATH_INFO: `/v3/services/network` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.562478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:19.564533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:19.567550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:19.573546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adc594-d3b2-43aa-8368-85ce64dc503a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a0aea076-8458-4290-94ea-571c5e921f66 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:19.580692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:19.581669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:19.587266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 380/774] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:19.588070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce244a3-2095-4491-ad63-51d1b8634472 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.588070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce244a3-2095-4491-ad63-51d1b8634472 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.588070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce244a3-2095-4491-ad63-51d1b8634472 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.588070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 381/775] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:19.593026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce244a3-2095-4491-ad63-51d1b8634472 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.623929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:19.624555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.624830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.625105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.625810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] RBAC: Adding query filter params (interface=public, service_id=2150ac84e0444780a880001ea4e49d33, region_id=RegionOne) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:19.627069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:19.628750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1123f-5862-4034-89c1-a41e83fa2194 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:19.638485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 395/776] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/endpoints?service_id=2150ac84e0444780a880001ea4e49d33&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:19.639696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0054-0954-4807-8316-a63cd930932a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.640029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0054-0954-4807-8316-a63cd930932a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.640305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf0054-0954-4807-8316-a63cd930932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.670367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-36bf0054-0954-4807-8316-a63cd930932a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:19.672392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-36bf0054-0954-4807-8316-a63cd930932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:19.725194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce244a3-2095-4491-ad63-51d1b8634472 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.758077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 396/777] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:19.759261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0054-0954-4807-8316-a63cd930932a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.766702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:19.768359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:19.770986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:19.771468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:19.772355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:19.772857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:19.774554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:19.784040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 382/778] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:19.790904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:19.829907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:19.940365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:19.957477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf0054-0954-4807-8316-a63cd930932a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:19.977720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954a1c2-9fa0-431b-a296-deb0132f5fd7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:20.025275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.026354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.026832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.027302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.028179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:20.029846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:20.042521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 397/779] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/services => generated 772 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:20.043985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.044900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.045403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.045953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.046857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] RBAC: Adding query filter params (type=network) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:20.047360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:20.048966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:20.055329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 383/780] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:20.062562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:20.103191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:20.264817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:20.275320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf42937-1d47-43fb-912f-16ffade2436a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:20.290107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03002d-1446-46f4-aee3-1d73ce267f2b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:20.327791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.328422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.328677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.328947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.329644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] RBAC: Adding query filter params (interface=public, service_id=5da5e0bf4e2d4eeb87bb7308a0ff1b29, region_id=RegionOne) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:20.329952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:20.331340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:20.345379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 398/781] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:20 2022] GET /identity/v3/endpoints?service_id=5da5e0bf4e2d4eeb87bb7308a0ff1b29&interface=public®ion_id=RegionOne => generated 194 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:20.345966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.346651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.346929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.347199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.352759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:20.372711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:20.378986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:20.379988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:20.380841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:20.394125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 384/782] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:20.396875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:20.403794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:20.507578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16da83-2377-4b0b-879a-96e98407a0a1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:20.546337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe0f86-6b48-4e0b-a09f-88125d25c6ec None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:20.559495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50672c0a-da73-418c-ba84-a729c870d8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.560656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50672c0a-da73-418c-ba84-a729c870d8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.561149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50672c0a-da73-418c-ba84-a729c870d8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.562115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50672c0a-da73-418c-ba84-a729c870d8d6 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.563236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50672c0a-da73-418c-ba84-a729c870d8d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:20.565483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50672c0a-da73-418c-ba84-a729c870d8d6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:20.578025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 399/783] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:20 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:20.578025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6df36d-7f61-4195-813b-4c35bee6e91b None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.578025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6df36d-7f61-4195-813b-4c35bee6e91b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.578025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6df36d-7f61-4195-813b-4c35bee6e91b None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.583456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 400/784] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:20.584769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.585053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.585327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.601969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629316-b290-4199-8853-1178875097f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.602619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629316-b290-4199-8853-1178875097f0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.602899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629316-b290-4199-8853-1178875097f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.603179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629316-b290-4199-8853-1178875097f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.603901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629316-b290-4199-8853-1178875097f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:20.604236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629316-b290-4199-8853-1178875097f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:20.611260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629316-b290-4199-8853-1178875097f0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:20.625793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 385/785] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:20.626644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:20.628222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:20.632388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629316-b290-4199-8853-1178875097f0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:20.653571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:20.781394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 401/786] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:20.799334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629316-b290-4199-8853-1178875097f0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:20.837370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:20.838083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:20.838429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:20.838754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:20.850178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:20.851633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:20.867339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:20.868324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:20.869209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:20.869209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:20.869209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:20.869209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 386/787] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:20.875681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:20.895724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.014611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b437e-f445-46e3-a008-7be52b07bc88 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.062241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.062779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.063060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.063334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.063987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:21.064281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.065655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.078132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 402/788] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:21.083565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.115221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.188006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.205652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.237370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992fc9e4-a482-47fd-a50f-2101101afdf3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.241573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc2b95-5077-4ef8-9b09-b560e526eb38 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.271305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46a845-2679-445d-9dcb-586b50aa0fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.272043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a845-2679-445d-9dcb-586b50aa0fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.272328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a845-2679-445d-9dcb-586b50aa0fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.272702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46a845-2679-445d-9dcb-586b50aa0fc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.277959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.278629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.278905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.279165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.283865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46a845-2679-445d-9dcb-586b50aa0fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:21.286900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.288337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-fd46a845-2679-445d-9dcb-586b50aa0fc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:21.289120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:21.290114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:21.290959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 403/789] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:21.292335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0fd70-4cfb-4f9f-809f-497a7bde1ddb None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.292335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0fd70-4cfb-4f9f-809f-497a7bde1ddb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.292695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0fd70-4cfb-4f9f-809f-497a7bde1ddb None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.297816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 404/790] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:21.299062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.299340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.299678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:21.306815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:21.309487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 387/791] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:21.311685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.322024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:21.323415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:21.394671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 405/792] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:21.395917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.412207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e1864-d0b3-4b0c-9923-47a676f7efa1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.429670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.448908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.449679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.449962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.450277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.450949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:21.451533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.452902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.465211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 388/793] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:21.471290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.486489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.547083 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e6da7-9645-41b6-8c9d-2adb94810b77 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.624262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.624803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.625085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.625352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.626109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:21.626410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.628109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.638237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a71aaa-c26c-4112-b13d-d66a5083ecaf None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.672595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 406/794] 10.222.0.50 () {56 vars in 1135 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:21.675485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.693129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.693766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.694078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.694361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.706869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.708476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:21.723790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:21.724987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:21.726099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:21.726099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:21.726099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:21.730741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 389/795] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:21.731753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.772892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.837469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80950524-4cb9-4c4e-ae57-5df4d19ca923 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.853656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5daec-0f5d-4287-aa5a-0ce2ccf4f90e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:21.869339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:21.887822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.888390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.888667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.888940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.896528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:21.897860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:21.898344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:21.899256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:21.900358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:21.901384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.901754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:21.903160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.903270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:21.922667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:21.923648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:21.924526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:21.924526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:21.924526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:21.924526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 407/797] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:21.924526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:21.925065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 390/797] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:21.926103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.035115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.046402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302aaf6d-4d11-4ec0-b306-5ccf54049427 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.052903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb54ac-641d-408b-b3d5-d00ba2eded73 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.089324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.090370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.090658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.091055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.095786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.096685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.097161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.097652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.099283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.101218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.110318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:22.114424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:22.115334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:22.116267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:22.118847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:22.120376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:22.121199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:22.121825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 391/798] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:22.121825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.123342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 408/799] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:22.124054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.165827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.232401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a2e30-6e45-4d8d-a44a-94e5b61da2a2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.239621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7e5e6-b47a-4ede-a280-68735adce3a6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.269234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.270350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.270846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.271324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.272240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:22.272798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.274478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.277572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.277915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.278244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.278522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.279254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:22.279573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.281013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.294227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 392/800] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:22.295419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.300833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 409/801] 10.222.0.50 () {56 vars in 1149 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:22.301674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.322361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.410120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e4dd0-0b20-48e0-a8c8-b133a22cb373 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.416249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.426436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37476-512b-4985-91f5-6035a84c8738 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.450033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.450885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.451414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.451895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.452804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:22.453301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.455275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.465475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 393/802] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:22.466764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.470534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.472036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.472323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.472606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.481704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.483185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:22.494605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:22.495960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 410/803] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:22.502404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.552427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.577504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e368d7-1e32-49de-b95d-db5f2a8fbb77 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.611055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.611686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.611909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fb186-2bc2-456b-95a6-e77c1dd0a55a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.612015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.612231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.622731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.624213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-171aebc1-328b-4384-9df6-0c8715c03a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:22.638463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:22.639612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:22.640726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:22.640726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:22.640726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:22.644176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 394/804] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:22.645413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a5a87-09f7-4255-ab3b-afef4a55986f None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.645698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a5a87-09f7-4255-ab3b-afef4a55986f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.645975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a5a87-09f7-4255-ab3b-afef4a55986f None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.649210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 395/805] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:22.649210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a5a87-09f7-4255-ab3b-afef4a55986f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.659279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.660110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.660538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.660975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.673325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.675129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:22.691414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:22.692397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:22.693272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:22.693272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:22.693272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:22.693272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 411/806] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:22.693503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.701813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.760033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a5a87-09f7-4255-ab3b-afef4a55986f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.789380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.799201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.799920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.800197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.800582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.801298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:22.801572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.803046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b789-076c-4202-85d7-aadb286139a0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.807934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ad1d5-6ff6-4464-9eae-3de25855ab48 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:22.818865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 396/807] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:22.820419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.820698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.820971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.840516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:22.841869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:22.845469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:22.846112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:22.846405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:22.846692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:22.847303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:22.847742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:22.849822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:22.865729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 412/808] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:22.866620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.930130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 397/809] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:22.930130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:22.961905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:22.986435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc184a1-edc8-44da-bb7b-46e227af1e98 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.028574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.029189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.029567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.029846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.030547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:23.030848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.032875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.038385 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bc245-7b04-4767-86f3-bc3c69409ef8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.050146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 413/810] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:23.051054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.061158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:23.082255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd49cf-596d-4ca5-8289-f7b8e5eb70fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.083005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd49cf-596d-4ca5-8289-f7b8e5eb70fd None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.083287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd49cf-596d-4ca5-8289-f7b8e5eb70fd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.083564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd49cf-596d-4ca5-8289-f7b8e5eb70fd None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/b8a56d415755492fae33fc13f57c8f3c/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.085496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd49cf-596d-4ca5-8289-f7b8e5eb70fd None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=b8a56d415755492fae33fc13f57c8f3c, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.087018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dd49cf-596d-4ca5-8289-f7b8e5eb70fd None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.100289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 398/811] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:48:22 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/b8a56d415755492fae33fc13f57c8f3c/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:23.101502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e5ee-5df6-4101-a2a8-b972b4183b78 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.101787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e5ee-5df6-4101-a2a8-b972b4183b78 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.102428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949e5ee-5df6-4101-a2a8-b972b4183b78 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.104516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 399/812] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:23.104868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e5ee-5df6-4101-a2a8-b972b4183b78 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.196597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9d834-4d9c-406e-8637-0737c1dec876 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.234566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:23.249666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.250625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.250625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.250881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.258105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949e5ee-5df6-4101-a2a8-b972b4183b78 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.259270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.261940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-da4cae14-8b34-4853-a997-4a60d81b82be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:23.274097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:23.275111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:23.276653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 414/813] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:23.277190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.278339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.278339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.298722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:23.300164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:23.305181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.305820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.306186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.306468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.313835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.315525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:23.328599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:23.329708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 400/814] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:23.330675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.368109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:23.382460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 415/815] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:23.383590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.450971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa1d5f-32c8-4ecb-b470-bd50f3ea1361 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.494124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.495733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.496387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.496859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.498243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.500326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.503192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3f74e-e7e3-4711-a599-94f5fd447ac8 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:23.510880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:23.511886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:23.517670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 401/816] 10.222.0.50 () {56 vars in 1129 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:23.518506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.531657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:23.556426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.557844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.558480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.559484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.560755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:23.561394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.564118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.584085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 416/817] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:23.585188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.627876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:23.631928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b67f7-3abf-460f-a5a6-769191b0c53b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.684627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.685819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.686831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.687327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.688565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:23.689267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.691420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.705485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e5654-4a99-49e8-bfbd-08e0284f0e68 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.711804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 402/818] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:23.712552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.745099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.745705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.745986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.746283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.746908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:23.747205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.748560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.753416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 417/819] 10.222.0.50 () {56 vars in 1124 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:23.754150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.804410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:23.872122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26546bf0-5f46-4576-a6a9-f78b45e1570c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.908360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.908993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.909769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.910069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] PATH_INFO: `/v3/services/network` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:23.910681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:23.912019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:23.922052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:23.923200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:23.924044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 418/820] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:23.924820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:23.950501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7b8ee-ddd2-4295-8417-68c0a2181562 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:23.987315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:23.988258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:23.988705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:23.989177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/8b1bc0d6bb8248cd86c8ac462709e326/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.005060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=8b1bc0d6bb8248cd86c8ac462709e326, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.006612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.030025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 403/821] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:48:23 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/8b1bc0d6bb8248cd86c8ac462709e326/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:24.030025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.033403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.045305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d8f8a-f793-4cbd-894f-76f086e77991 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.047166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.094152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.094894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.095170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.095443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.136431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c706a-029c-4d8e-a71f-b2543c02dec1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.179808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.180729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.181172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.181722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.182934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:24.183384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.184937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.189972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.195368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 404/822] 10.222.0.50 () {56 vars in 1124 bytes} [Sat Sep 24 04:48:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:24.195368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.198256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.285669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.312620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104a8d-1873-4891-8a8f-2e37ef918dff None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.325407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 419/823] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:48:24.326648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.329914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.360755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.361768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.362094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.362377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.363188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:24.363588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.365226 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.372046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 405/824] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:24.372872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.445495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fe720-f05b-46ce-8a31-4d2f798f9bbb None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.482196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.483158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.483627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.484087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.484966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.486473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.509260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a2c0f-7f76-4805-8151-31c8107a54a2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.515918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 420/825] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:24.517103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.551566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.556875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.558215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.558890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.559258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.571400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.572336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:24.591073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:24.592256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:24.593349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:24.593349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:24.593349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:24.594855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 406/826] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:24.603098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.604925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.715451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c0e05-a1ae-4edd-ab21-d28c6a7bfe79 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.760825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.761875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.762300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.762710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.763477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] RBAC: Adding query filter params (type=network) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:24.763917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.764858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6dedf-93b4-418a-b40e-2cf1292a0e1d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.765600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.771145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 421/827] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:24.780087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.813741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.814572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.814847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.815116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.815901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:24.816207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.818201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.832805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 407/828] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:24.839112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:24.903712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.906277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4d4c8-4564-41f2-bcfe-7319e2e0bb57 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.933301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:24.954187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:24.954816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:24.955098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:24.955372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:24.956051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:24.957438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:24.988021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80eda9-1457-4aa8-8379-2401d4f46f7a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:24.996119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 422/829] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:24 2022] POST /identity/v3/endpoints => generated 337 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:25.051743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:25.052389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:25.052669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:25.052940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:25.060632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:25.062117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:25.074537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:25.075610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:25.076515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 408/830] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:25.081790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:25.208992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0b8f7-21f5-46f9-a330-18404de0e62a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:25.247666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:25.248346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:25.248631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:25.248908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:25.249616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:25.249921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:25.251388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00d92e-0bd5-466a-98ee-d3502da3c9d3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:25.264410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 423/831] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:25.270374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:25.384661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:25.388836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b75827-f187-4cd7-ae58-ae8ede827623 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:25.425355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6886ee-0592-4d58-a338-992a48e38b63 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:25.425715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6886ee-0592-4d58-a338-992a48e38b63 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:25.425911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6886ee-0592-4d58-a338-992a48e38b63 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:25.427832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 424/832] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:25.428738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:25.429452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:25.429735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:25.430029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:25.430771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=2451d53fb9974222827b6520b8bce5f3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:25.431071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:25.432699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:25.432897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:25.433116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:25.433386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:25.439152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 409/833] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:25 2022] GET /identity/v3/role_assignments?user.id=2451d53fb9974222827b6520b8bce5f3&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:25.453661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:25.454934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:25.489862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:25.535862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 425/834] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:25.814263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:25.925513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9a041-db20-4c11-a8dc-d131d1ca3d96 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:25.960569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:25.961584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:25.962093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:25.962563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:25.974034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:25.979402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:25.980291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:25.981125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 410/835] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:25.988267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.089903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be55e8-fa10-4d6b-986e-1d7f4603d519 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:26.125513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:26.126520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.127259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.127723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.128565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:26.129075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:26.130639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:26.140034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 426/836] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:26.145905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.174269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:26.258507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33311b49-9159-4bc8-90b8-35e330f368ea None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:26.294708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:26.295705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.296183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.296650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.308388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:26.310231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-120699bb-7a19-40ef-b4b8-5e17b77bfd9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:26.325762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:26.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:26.327668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:26.327668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:26.327668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:26.329861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 411/837] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:26.335797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.369447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:26.447912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978b23a-90ba-4152-84e3-160c31bbeb55 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:26.481723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:26.482356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.482630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.482898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.483531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:26.483829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:26.485181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:26.488777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdbf0e-655e-4ee6-aec7-91c6fed58ee7 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.489058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdbf0e-655e-4ee6-aec7-91c6fed58ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.489328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdbf0e-655e-4ee6-aec7-91c6fed58ee7 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.491981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 412/838] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:26.496427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.496711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.496985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.497873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 427/839] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:26.503409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.517797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:26.519102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:26.596593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:26.607552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c82f8c-63af-4a4e-93c1-7599bc3f1f78 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:26.610083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 413/840] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:26.648682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:26.649317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.649591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.649861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] PATH_INFO: `/v3/users/swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.663360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:26.664800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:26.676892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:26.677847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:26.678888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 428/841] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:26.684746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.711308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:26.786537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e330d-749e-49d4-a316-6812f23cf021 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:26.821563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:26.822190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:26.822465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:26.823257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:26.823984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:26.824329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:26.825811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:26.843844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 414/842] 10.222.0.50 () {56 vars in 1135 bytes} [Sat Sep 24 04:48:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:26.849399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.888792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:26.959939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:26.962668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1d029-00a8-4cef-b566-a001c54154df None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.001333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.002466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.002888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.003326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.003647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4dfb3-d8bc-4f51-931c-047d7d2b8d38 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.025334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.048710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.049564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.050022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.050438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.063351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:27.069653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:27.070808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:48:27.071818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:27.075294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 415/843] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:48:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:27.087005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.093086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.094825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.218040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 429/844] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:48:27.219043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.232223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cddaa1-548a-4e69-8ba7-2f34ce4c2f29 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.267065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.272063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.282442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.283088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.283360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.283662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.284355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:27.284672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.286145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dbfd0-5c4f-4778-b28b-03d99101cb07 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.294383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 416/845] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:27.295621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c1fa-845d-4272-9482-d40b30806d74 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.295910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c1fa-845d-4272-9482-d40b30806d74 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.296183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c1fa-845d-4272-9482-d40b30806d74 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.297524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 417/846] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:27.299780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c1fa-845d-4272-9482-d40b30806d74 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.334033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5baadd-d720-4bfc-8ba4-b76fefd6ae47 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.373485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.374140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.374423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.374695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.386179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.387640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-785ff148-66f0-45d7-a620-452fc150dad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:27.404894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:27.405862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 430/847] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.411768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.412202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c1fa-845d-4272-9482-d40b30806d74 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.427240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:27.428831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:27.448339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.448968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.449244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.449514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.461182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.462689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:27.477223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:27.478193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:27.479205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:27.479205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:27.479205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:27.479205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 418/848] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:27.479205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.504190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 431/849] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:27.505328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.536989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.542646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.623178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4316f1-f0f1-4927-9d2e-d056bc1c843a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.627921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075d380-6e64-43f4-a2ee-28d192723cac None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.665172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.666132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.667238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.667699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.668609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:27.669134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.670847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.673085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.673754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.674543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.674824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.675470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:27.675772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.677119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a02b2e-fa59-4201-a93d-9c1106386d9b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.690497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 419/850] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:27.691657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 432/851] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:27.692111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.692973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44219b2f-0000-4e56-ac88-8b0586c7fea8 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.693263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44219b2f-0000-4e56-ac88-8b0586c7fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.693544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44219b2f-0000-4e56-ac88-8b0586c7fea8 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.695031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 433/852] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:27.695727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44219b2f-0000-4e56-ac88-8b0586c7fea8 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.793941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.796854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:27.801093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e94ea3-8cbd-48a4-9522-d5c165996a0e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.804191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44219b2f-0000-4e56-ac88-8b0586c7fea8 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:27.834742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.835370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.835650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.835926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.837634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:27.838802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.839095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.839358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.843165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.844592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.846366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:27.847772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-199d7e0d-79de-48bf-9567-52b707d7cbda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:27.856114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:27.856982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:27.857866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 420/853] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:27.858751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:27.859019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:27.859292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:27.859716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:27.860572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:27.861507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:27.863003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 434/854] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:27.863806 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.877280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:27.878441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:27.947660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 421/855] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:27.949044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:27.969191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a0d937-ca0f-4520-90a3-75c9aaa2bf46 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.008099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.008698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.008975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.009407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.009827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.011826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:28.021246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:28.022433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:28.023461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:28.027262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 435/856] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:28.028024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.042589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.048174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.055210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37131322-22a9-46e9-a1c4-4affc12b9e9c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.090185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.090782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.091231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.091511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.092196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.092493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.094026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.107523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 422/857] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:28.108656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.148723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbb192-8633-496f-9b9f-ede81d2f9efc None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.188804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.189709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.190228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.190759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.191744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.192219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.194484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.219869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 436/858] 10.222.0.50 () {56 vars in 1149 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:28.220581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c792c-27c3-4b4a-b125-3990ce1951d3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.221126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.256141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.256792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.257067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.257337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.257961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.258287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.259658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.264124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 423/859] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:28.264885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.279145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.315174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.340181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51354c30-fd72-4570-a384-64f72e0c1fdd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.362940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f468196-a5db-456c-be50-471b061ca1ef None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.372481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.373081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.373358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.373630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.374368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=b8a56d415755492fae33fc13f57c8f3c) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.374672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.376115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.381909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 437/860] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/role_assignments?user.id=b8a56d415755492fae33fc13f57c8f3c&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:28.382699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.401625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.402217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.402503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.402782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.403378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.404777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:28.414325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:28.415336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:28.416310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 424/861] 10.222.0.50 () {56 vars in 1132 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:28.418442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.503996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.511665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd652d7-1be4-4f21-aece-b64a1e986a9b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.529739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0bbfa-5dbb-48d9-8d35-52cf8e1b7f14 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.549687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.550333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.551536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.551536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.561655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.571681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.572290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.573820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.574123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.574760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.575054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.576414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.581123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 425/862] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:28.581894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.667132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.668774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.749572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb14f08-119d-4c33-9b53-125c89a638a0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.795209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.795904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.796247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.796585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.797012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 438/863] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:48:28.797410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.797838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.799832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.804617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.809812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 426/864] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:28.816297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:28.820078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.854502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:28.924331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b80317-12a4-4b40-aafa-cfc502da57a1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.936380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d9e5c-8798-4878-9cef-848f8851c535 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:28.960872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.962038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.962514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.962974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.964363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:28.964898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:28.966487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:28.971618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 427/865] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:28.984668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:28.985780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:28.986297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:28.986764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:28.999052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:29.000997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:29.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:29.019643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:29.020519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:29.020519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:29.020519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:29.022728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 439/866] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:29.028746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:29.111374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:29.151066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabc485-82b2-485d-8cbf-dc08b260e65a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:29.187216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:29.187759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:29.188037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:29.188307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:29.189024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:29.189323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:29.190750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:29.204458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 428/867] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:29.209853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:29.310265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:29.320006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec553d-5eee-468e-ba2c-8c3704e28231 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:29.359862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:29.360509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:29.361068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:29.361344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:29.368633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:29.370667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:29.383482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:29.384824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:29.385783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 440/868] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:29.392454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:29.497333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeed0d2-21d9-4045-9644-fb8cb913d18a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:29.527740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:29.538346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:29.539310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:29.539830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:29.540303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:29.541210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:29.541776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:29.543480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb973b-fe82-483f-8fb8-b65f20000055 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:29.556334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 429/869] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:29.563041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:29.671511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e67c7c-c408-433b-a802-97811779b547 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:29.707458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:29.708070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:29.708348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:29.708623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:29.709347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=8b1bc0d6bb8248cd86c8ac462709e326) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:29.709705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:29.711304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2f57e-f6ff-466e-83f3-7d926bc69064 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:29.717669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 441/870] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:29 2022] GET /identity/v3/role_assignments?user.id=8b1bc0d6bb8248cd86c8ac462709e326&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:30.610210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77a3b0-6762-4ddd-ab8d-8e43c86482fd None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:30.611336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77a3b0-6762-4ddd-ab8d-8e43c86482fd None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:30.611837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77a3b0-6762-4ddd-ab8d-8e43c86482fd None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:30.613657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 430/871] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:30.618788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfcf5-6360-47a5-9ee8-4155f080c26f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:30.619225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfcf5-6360-47a5-9ee8-4155f080c26f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:30.619310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cfcf5-6360-47a5-9ee8-4155f080c26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:30.638520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1f9cfcf5-6360-47a5-9ee8-4155f080c26f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:30.639706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1f9cfcf5-6360-47a5-9ee8-4155f080c26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:30.649623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:30.715109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 442/872] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:30.820565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f3db-e8b0-41eb-9c13-ba16bec414a7 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:30.820905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f3db-e8b0-41eb-9c13-ba16bec414a7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:30.821115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f3db-e8b0-41eb-9c13-ba16bec414a7 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:30.823254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 431/873] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:30.830427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:30.830824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:30.831203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:30.849475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:30.850688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:30.917932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 443/874] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:30.989109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f3db-e8b0-41eb-9c13-ba16bec414a7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:31.094897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f3db-e8b0-41eb-9c13-ba16bec414a7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:31.130502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:31.131498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.131988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.132456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.133234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:31.134803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:31.143878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:31.144918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:31.145822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:31.153560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 432/875] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:31.155209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:31.191313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:31.256026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:31.279012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:31.289385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb171-e679-4a2e-b6f1-da8b5198a346 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:31.327406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946a04f-294d-43fa-a608-f0303c795d56 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:31.328452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:31.329074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.329346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.329616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.330342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:31.330637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:31.331984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b34734-9a52-40d0-9545-65c3feeed2be None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:31.336610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 444/876] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:31.337849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211611-dfb4-472e-897a-c5372837e2d7 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.338166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211611-dfb4-472e-897a-c5372837e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.339046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7211611-dfb4-472e-897a-c5372837e2d7 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.340297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 445/877] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:31.347097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211611-dfb4-472e-897a-c5372837e2d7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:31.372021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1472b9-c087-46c5-baed-4dcd6cfa7f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:31.373252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1472b9-c087-46c5-baed-4dcd6cfa7f75 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.373701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1472b9-c087-46c5-baed-4dcd6cfa7f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.374175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1472b9-c087-46c5-baed-4dcd6cfa7f75 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.375016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1472b9-c087-46c5-baed-4dcd6cfa7f75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:31.376549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1472b9-c087-46c5-baed-4dcd6cfa7f75 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:31.394913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 433/878] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:31 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:31.396445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.397290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.397729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.427030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:31.428927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:31.482900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7211611-dfb4-472e-897a-c5372837e2d7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:31.526330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:31.527292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.527767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.528229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.529054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:31.529559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:31.531163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608917f8-663b-45e1-b87a-3f9f53b15fc2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:31.533252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:31.533575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 434/879] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:31.538407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 446/880] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:31.839867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:31.884918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:31.953889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c52770-c44c-435e-91ff-4062c1d1ef6a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:31.990621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998f46b-4013-4afa-a9bc-feadad78bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:31.991597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f46b-4013-4afa-a9bc-feadad78bc68 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:31.992100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f46b-4013-4afa-a9bc-feadad78bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:31.992581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998f46b-4013-4afa-a9bc-feadad78bc68 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:31.993500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f46b-4013-4afa-a9bc-feadad78bc68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:31.995197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998f46b-4013-4afa-a9bc-feadad78bc68 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:32.014162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 435/881] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:31 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:32.042261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8e13-4c60-4029-b0b5-58b96c986a32 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:32.042815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8e13-4c60-4029-b0b5-58b96c986a32 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:32.042815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8e13-4c60-4029-b0b5-58b96c986a32 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:32.044736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 447/882] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:32.051876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:32.052202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:32.052430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:32.077202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:32.078459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:32.165603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 436/883] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:32.438507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a8e13-4c60-4029-b0b5-58b96c986a32 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:32.541564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a8e13-4c60-4029-b0b5-58b96c986a32 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:32.576217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:32.576896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:32.577178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:32.577454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:32.587622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:32.593070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:32.594035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:32.594873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 448/884] 10.222.0.50 () {56 vars in 1120 bytes} [Sat Sep 24 04:48:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:32.601968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:32.635497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:32.710499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f43c-ea18-4bf4-a10b-3773763f9c1d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:32.750543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:32.751156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:32.751435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:32.751705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:32.752633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:32.752935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:32.754312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:32.763796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 437/885] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:48:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:32.771301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:32.836858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:32.878093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d99b1-03e6-4c65-810f-b71aa2910b00 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:32.912762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:32.913385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:32.913666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:32.913925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:32.924499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:32.925987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:32.940435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:32.941395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:32.942429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:32.942429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:32.942429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:32.942429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 449/886] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:32.949379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:33.056395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1c1b6-731a-42fc-8e69-1e590e41b951 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:33.070355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:33.091457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:33.092064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.092342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.092610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.093256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:33.093567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:33.094994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:33.107271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 438/887] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:33.112690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:33.218611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc678f5b-94bd-4631-8f85-44c977de300d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:33.229233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:33.253654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:33.254326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.254604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.254917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.262737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:33.264174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:33.275597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:33.277170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:33.278151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 450/888] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:48:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:33.284925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:33.391976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec70e5c-44d6-4402-9a6b-3f9295d85a37 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:33.451439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:33.452912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.453700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.454489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.456083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:33.456875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:33.459886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268e9f0-2ef9-4c1b-bf0f-9ff66e057939 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:33.491443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 439/889] 10.222.0.50 () {56 vars in 1149 bytes} [Sat Sep 24 04:48:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:33.503334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:33.608950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041d2c4-2537-4315-b9b9-443ecc5ef0c6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:33.643553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7888e5-e84c-4a12-8878-1cee8a965e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:33.644083 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7888e5-e84c-4a12-8878-1cee8a965e7d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.644354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7888e5-e84c-4a12-8878-1cee8a965e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.644625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7888e5-e84c-4a12-8878-1cee8a965e7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.693574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:33.706048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b1d69-b508-489f-9236-ad7f943f1490 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.706350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b1d69-b508-489f-9236-ad7f943f1490 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.706603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b1d69-b508-489f-9236-ad7f943f1490 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.707953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 440/890] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:33.713684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.714028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.714264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.722258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7888e5-e84c-4a12-8878-1cee8a965e7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:33.724169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7888e5-e84c-4a12-8878-1cee8a965e7d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:33.733556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:33.734791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:33.736937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:33.805255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 441/891] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:33.840896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 451/892] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:33 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:48:33.848767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:33.853223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b76ff-e051-4d18-a4c1-3f815580733a None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.853515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b76ff-e051-4d18-a4c1-3f815580733a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.853789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b76ff-e051-4d18-a4c1-3f815580733a None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.855933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 452/893] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:33.860983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:33.861272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:33.861544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:33.880350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:33.881550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:33.951493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:33.961665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239b6a9-8266-48e7-9fdf-fdf559a65c45 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:33.964680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 453/894] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:34.003445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.004080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.004354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.004623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.011427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.015551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.017009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:34.032400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:34.033349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:34.034253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:34.034253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:34.034253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:34.034253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 442/895] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:34.040843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.073932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.165466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c952f-ab0c-4d8a-9055-e52a928c8608 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.191731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87affd2-ae1a-46d2-a6d5-75ab9e852992 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.206059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.207034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.207429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.207802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.208656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:34.209052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.210667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.230421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 454/896] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:34.231671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.233826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.234889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.235304 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.236568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.237372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.239026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:34.251572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:34.253096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:34.254135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:34.261749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 443/897] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:34.267248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.308365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.337227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.355208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439355c0-0fd6-4530-9667-6146e2e1a082 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.386563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c6f63-ff2a-400c-bb2e-7afb58b4d83f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.389820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.390479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.390756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.391052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.397451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.398914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:34.410560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:34.411634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:34.412729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 455/898] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:34.413356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.421953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.422577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.422872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.423137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.423762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:34.424057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.425389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e07fa9-5318-4d74-92f2-edcac6bcc040 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.430011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 444/899] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:34.431171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5b102-bc09-46d2-8163-5e3ed2668fd0 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.431461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5b102-bc09-46d2-8163-5e3ed2668fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.431715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee5b102-bc09-46d2-8163-5e3ed2668fd0 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.434127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 445/900] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:34.434127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5b102-bc09-46d2-8163-5e3ed2668fd0 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.513829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ce587-a1b2-4b6b-8385-904c11c30400 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.532765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.534086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee5b102-bc09-46d2-8163-5e3ed2668fd0 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.547365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.547972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.548247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.548518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.549165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.550559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.550932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.564903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 456/901] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:34 2022] POST /identity/v3/services => generated 241 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:34.565691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.576516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.577148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.577429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.577688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.578418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:34.578720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.580191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9d272-5a82-4fcc-86df-674f786f0667 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.594382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 446/902] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:34.596243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.596786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.597260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.616287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:34.617537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:34.692569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0dc5a7-dfe7-4ec7-825a-b4256d1d15fa None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.699049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 447/903] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:34.700264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.730469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.731089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.731370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.731642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.732496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:34.732618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.733939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.740439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 457/904] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:34.745949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.801562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.824574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8602954-ba5a-42c5-b7a3-3a93d71de7a5 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.827847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:34.873729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.874418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f129f60-a2a6-434a-ade0-0944be22e4fc None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:34.874847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.875331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.875790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.878059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] RBAC: Adding query filter params (role.id=3e2c1a2d80234e7fbf13a69c075de868, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=8b1bc0d6bb8248cd86c8ac462709e326) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:34.878548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.880153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.887627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 448/905] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/role_assignments?user.id=8b1bc0d6bb8248cd86c8ac462709e326&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=3e2c1a2d80234e7fbf13a69c075de868 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:34.921048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:34.921675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:34.922035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:34.922318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:34.923083 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] RBAC: Adding query filter params (interface=public, service_id=21627bf27dc3432c9c26bb204e460a9e, region_id=RegionOne) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:34.923394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:34.924801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:34.931362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 458/906] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/endpoints?service_id=21627bf27dc3432c9c26bb204e460a9e&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:34.937848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:34.982028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:35.059072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc79c1-0871-4176-a76d-01d0237e1a53 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:35.100430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:35.100664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:35.100755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:35.101009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:35.101623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:35.103022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:35.110843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 449/907] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:48:35.124097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287b7e5-fe0c-43b1-8e93-f63357585d48 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:35.136737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:35.168009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:35.169473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:35.169944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:35.170432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:35.171240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:35.172871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:35.183165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:35.184198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:35.185085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:35.187626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 459/908] 10.222.0.50 () {56 vars in 1132 bytes} [Sat Sep 24 04:48:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:35.195381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:35.231428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:35.309298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c755b-209e-4ebc-a1c6-8386f41331d5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:35.346733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:35.347393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:35.347680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:35.347959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:35.348603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:35.348903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:35.350286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:35.355656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 450/909] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:35.361353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:35.479236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb00e76-f62e-49e8-950c-09a4899aacc2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:35.515675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:35.516285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:35.516567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:35.516839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:35.517477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:35.517781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:35.519178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:35.525041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 460/910] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:35.535461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:35.633407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:35.641011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e0172-7efc-4cce-8633-6901a60059ea None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:35.685382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:35.686011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:35.686302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:35.686571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:35.687277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] RBAC: Adding query filter params (interface=public, service_id=b0f3b0ac43694d95b1a23bccbd7546bd, region_id=RegionOne) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:35.687580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:35.688934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56d485-eecd-40cc-a279-93fbf23d4a7b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:35.697177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 451/911] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:35 2022] GET /identity/v3/endpoints?service_id=b0f3b0ac43694d95b1a23bccbd7546bd&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:35.702808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:35.741306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:35.809451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147b18b-d3c8-423c-a930-aec18fe8cd5f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:35.846327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce867d5e-a89c-4157-96eb-3f1033932e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:35.847301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce867d5e-a89c-4157-96eb-3f1033932e9b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:35.847780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce867d5e-a89c-4157-96eb-3f1033932e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:35.848251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce867d5e-a89c-4157-96eb-3f1033932e9b None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:35.849072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce867d5e-a89c-4157-96eb-3f1033932e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:35.850741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce867d5e-a89c-4157-96eb-3f1033932e9b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:35.857520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 461/912] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:35 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:48:36.860905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcd3da-b434-418a-ac85-8c3acae2e3e1 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:36.861436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcd3da-b434-418a-ac85-8c3acae2e3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:36.861436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fcd3da-b434-418a-ac85-8c3acae2e3e1 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:36.863083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 452/913] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:36.868318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be829e5-dc8c-4899-a377-94df1856913b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:36.868721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be829e5-dc8c-4899-a377-94df1856913b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:36.868885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be829e5-dc8c-4899-a377-94df1856913b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:36.887747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6be829e5-dc8c-4899-a377-94df1856913b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:36.888930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6be829e5-dc8c-4899-a377-94df1856913b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:36.956255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 462/914] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:37.201021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcf89c-9dee-46bc-ba91-9833f902d92e None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.201379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcf89c-9dee-46bc-ba91-9833f902d92e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.201599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcf89c-9dee-46bc-ba91-9833f902d92e None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.203270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 453/915] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:37.209203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930937cb-cb37-4f5a-b00f-080836501ffc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.209203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930937cb-cb37-4f5a-b00f-080836501ffc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.209203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930937cb-cb37-4f5a-b00f-080836501ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.229970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-930937cb-cb37-4f5a-b00f-080836501ffc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:37.231623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-930937cb-cb37-4f5a-b00f-080836501ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:37.236014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcf89c-9dee-46bc-ba91-9833f902d92e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.259489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:37.304927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 463/916] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:37.348797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcf89c-9dee-46bc-ba91-9833f902d92e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:37.351794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a93023-4d49-46d9-a92c-2488b1503a5c None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.352080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a93023-4d49-46d9-a92c-2488b1503a5c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.352354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a93023-4d49-46d9-a92c-2488b1503a5c None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.357421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 464/917] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:37.358815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.359091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.359359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.377194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:37.378757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:37.384828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:37.385974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.386277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.386543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.387520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:37.389227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:37.400021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:37.401069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:37.401901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.402664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 454/918] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:37.408384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.438125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:37.466196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 465/919] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:37.521652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf2133-867d-42d9-acde-9cba310dd5c5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:37.556842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:37.557899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.558424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.558886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.559698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:37.560234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:37.561786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:37.567102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 455/920] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:37.573543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.578335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.681714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:37.684599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:37.693683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290e601e-d26a-4565-aab4-c873ae0183a8 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:37.699459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82631680-2548-43c7-aba7-d0204b22c1ed None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:37.729865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:37.730839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.731311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.731799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.732613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:37.733103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:37.734681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:37.737674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:37.738345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.738625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.738878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.741066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 466/921] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:37.747543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.755167 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:37.759936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:37.761131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.762098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 456/922] 10.222.0.50 () {56 vars in 1120 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:37.762867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.857857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d64416-0929-42ca-bb97-c91094163912 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:37.866744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e93e0d-f3a0-45bb-afa5-6bcaeac8a5e3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:37.892676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:37.893818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.894262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.894532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.895223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] RBAC: Adding query filter params (interface=public, service_id=f66195d944b045f3b5dee34e0c75b463, region_id=RegionOne) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:37.895521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:37.896866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:37.902703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 467/923] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/endpoints?service_id=f66195d944b045f3b5dee34e0c75b463&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:37.903704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:37.904339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:37.904619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:37.904888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:37.905472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:37.906873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:37.907224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:37.916037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:37.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 24 04:48:37.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:37.922106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 457/924] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:37.922937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:37.928077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:37.939273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.016405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c9695-60ee-47ac-ae72-b4931c9fbd94 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.037422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26c72a-3639-44ab-812c-04665d36efe4 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.052425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.053029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.053303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.053596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.054245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:38.054541 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.056124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.066214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 468/925] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:38.066214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.075719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5933c5e-f108-4a9a-98c4-9ae71c56c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.076318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5933c5e-f108-4a9a-98c4-9ae71c56c171 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.076590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5933c5e-f108-4a9a-98c4-9ae71c56c171 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.076858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5933c5e-f108-4a9a-98c4-9ae71c56c171 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.077452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5933c5e-f108-4a9a-98c4-9ae71c56c171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.078830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5933c5e-f108-4a9a-98c4-9ae71c56c171 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.084491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 458/926] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:37 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:48:38.085594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c08d-f2e9-432a-864f-670a314c0321 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.085868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c08d-f2e9-432a-864f-670a314c0321 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.086162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160c08d-f2e9-432a-864f-670a314c0321 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.087346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 459/927] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:38.088048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160c08d-f2e9-432a-864f-670a314c0321 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.174802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.177158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.197633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1d937-62fa-4781-b420-03fba7a05b96 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.205429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160c08d-f2e9-432a-864f-670a314c0321 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.241142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.241632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.242256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.242371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.242557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.242823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.243076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.243486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.244249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:38.245429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.247182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8456b-8eb0-4f0c-b332-93dc9965df8a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.253779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 469/928] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:38.255314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323898-a066-4a22-8486-d123af9da002 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.255597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323898-a066-4a22-8486-d123af9da002 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.255867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323898-a066-4a22-8486-d123af9da002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.259298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.260788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:38.277393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:38.278421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:38.279295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:38.279295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:38.279295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:38.279295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 460/929] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:38.279295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.280795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e0323898-a066-4a22-8486-d123af9da002 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:38.282162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e0323898-a066-4a22-8486-d123af9da002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:38.363088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 470/930] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:38.363744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323898-a066-4a22-8486-d123af9da002 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.384669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10c14c-6b2e-4738-aca5-a183d767986d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.420512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.431900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.432845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.433277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.433709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.435042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:38.435629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.437560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.444925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 461/931] 10.222.0.50 () {56 vars in 1114 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:38.450773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.461048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.496667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323898-a066-4a22-8486-d123af9da002 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.540160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.540697 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.540979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.541252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.542484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:38.542787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.544270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.558870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 471/932] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:38.563727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.583530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399acc1-b1a6-4a42-ad0a-523d6bab2c78 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.618645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.620172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.620652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.621114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.621970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.623550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.648482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 462/933] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:38.650124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.682811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d286e9-9034-44b9-b7ed-a38770e48f25 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.719480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.738255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:38.744530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.745412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.745742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.746054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.756367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.757965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:38.772801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:38.773795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:38.774780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:38.780016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 472/934] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:38.781462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.786755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f271757-ca59-4c0d-9f49-3148b0b7d813 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.834977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.835546 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.835816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.836084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.836671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.838059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:38.848289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:38.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:38.850268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 463/935] 10.222.0.50 () {56 vars in 1132 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:38.857249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.908782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00485c08-0aac-4118-b1bf-5cf63b93f2ad None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:38.951836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:38.952590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:38.952965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:38.953330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:38.954249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:38.954647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:38.956628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:38.982085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 473/936] 10.222.0.50 () {56 vars in 1149 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:38.985742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:38.993320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead4affb-8cc4-4784-8f1b-ca752806f54e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.011312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:39.038824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.039438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.039720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.040308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.040956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:39.041260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.043499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.049608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 464/937] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:39.054423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:39.055366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:39.113265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a3278-b2f3-4de4-bf3d-f0431c2d340f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.160954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.162112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.162630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.163117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.178884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.180618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.193210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872697a-5fef-4af4-9a99-13ab4dfd8d0b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:39.202377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:39.203341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:39.204212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:39.204212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:39.204212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:39.208503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 474/938] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:39.209562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:39.233004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.233979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.234485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.234946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.235860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:39.236365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.237938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.243840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 465/939] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:39.250515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:39.304003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:39.332162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:39.336372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72a9ac-469e-4202-ac16-dcebda71c949 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.379284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69f77-7879-4f68-af35-3bb036975dad None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.383262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.383881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.384162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.384486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.385142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:39.386012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.387401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.406216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 475/940] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:39.407579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:39.418672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.419616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.420102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.420632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.421495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.423172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.450064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 466/941] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:39.534782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fd4f1-6067-407e-900b-215209391886 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.580861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.581885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.582429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.582893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.591048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.592905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:39.606559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:39.607504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:39.608423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:39.610992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 476/942] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:39.619135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:39.730628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:39.738203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572caef-845a-456c-9def-530b45a1bc04 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.779309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.779918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.780191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.780456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.781193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:39.781497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.782942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd113a7-e8a2-46f1-886f-15dc1cafdbf4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.796918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 467/943] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:39.802867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:39.824862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:39.912772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edc5d9-c2f8-4870-aad6-eec493fe03ad None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:39.951757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fde710-6deb-4287-bd85-152dee2329cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:39.952570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fde710-6deb-4287-bd85-152dee2329cf None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:39.952854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fde710-6deb-4287-bd85-152dee2329cf None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:39.953224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fde710-6deb-4287-bd85-152dee2329cf None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/8b1bc0d6bb8248cd86c8ac462709e326/roles/3e2c1a2d80234e7fbf13a69c075de868` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:39.955416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fde710-6deb-4287-bd85-152dee2329cf None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=8b1bc0d6bb8248cd86c8ac462709e326, role_id=3e2c1a2d80234e7fbf13a69c075de868)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:39.956846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fde710-6deb-4287-bd85-152dee2329cf None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:39.972228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 477/944] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:48:39 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/8b1bc0d6bb8248cd86c8ac462709e326/roles/3e2c1a2d80234e7fbf13a69c075de868 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:48:40.299736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79a6b-12f5-46fc-bc58-855655cb2c9b None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:40.300283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79a6b-12f5-46fc-bc58-855655cb2c9b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:40.300283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad79a6b-12f5-46fc-bc58-855655cb2c9b None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:40.301860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 468/945] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:40.307397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:40.307687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:40.307970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:40.327274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:40.328477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:40.404199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 478/946] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:40.674972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad79a6b-12f5-46fc-bc58-855655cb2c9b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:40.791069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad79a6b-12f5-46fc-bc58-855655cb2c9b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:40.829358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:40.830340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:40.830780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:40.831205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:40.831936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:40.833488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-49194502-f26f-431f-8f59-afe52b27d1b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:40.843459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:40.844728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:40.845773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:40.848382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 469/947] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:40.854937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:40.864540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:40.974349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2ef05-2f6d-4410-9c92-7902c8f8d440 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:40.974672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2ef05-2f6d-4410-9c92-7902c8f8d440 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:40.974774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32673fcc-0de5-4195-9325-a1abd0b8ac9f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:40.975170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2ef05-2f6d-4410-9c92-7902c8f8d440 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:40.976631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 470/948] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:40.982062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:40.982345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:40.982614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.001794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:41.003348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:41.013598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:41.014342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.014619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.014884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.015618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:41.015917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:41.017260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:41.020096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:41.023586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 479/949] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:41.029255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:41.098040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 471/950] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 24 04:48:41.120921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:41.154614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137f8016-dc02-4b20-bb5a-fe2b4cf79037 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:41.188744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:41.189681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.190213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.190711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.191582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:41.192078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:41.193636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:41.199611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 480/951] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:41.205486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:41.312031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7132f94-ad22-4a37-a7dd-020830cb7fc8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:41.349097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:41.350098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.350612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.351079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.351948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:41.353505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:41.363015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:41.382962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 472/952] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:41.500378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e67bc-f50b-4e90-ad1d-4990633445f7 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:41.539947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:41.547702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:41.549168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.549876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.550498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.551409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:41.553074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:41.556076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING keystone.server.flask.common [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 24 04:48:41.578668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.sql.core [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 04:48:41.578668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 04:48:41.578668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [parameters: {'id': '047e397ab8334c70a1e57f94a0441818', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 04:48:41.578668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 047e397ab8334c70a1e57f94a0441818.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 047e397ab8334c70a1e57f94a0441818. Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application conn.query(q) Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 04:48:41.596597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result.read() Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 24 04:48:41.597483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application next(self.gen) Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application yield resource Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application next(self.gen) Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise self.value Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application session.commit() Sep 24 04:48:41.598499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.session.flush() Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application compat.raise_( Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 04:48:41.599380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 04:48:41.600290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application conn.query(q) Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result.read() Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 04:48:41.601067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [parameters: {'id': '047e397ab8334c70a1e57f94a0441818', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:41.601962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:41.608182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 047e397ab8334c70a1e57f94a0441818. Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:41.609284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 481/953] 10.222.0.50 () {60 vars in 1140 bytes} [Sat Sep 24 04:48:41 2022] POST /identity/v3/projects => generated 262 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:41.612212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:41.624674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:41.725204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5a330-f82c-4c22-b72d-91707ed4b098 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:41.765956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:41.766659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.766939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.767213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.775205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:41.776758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-1ab4eede-cd22-4387-a44a-72569db3eb40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:41.789519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:41.790594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:41.792568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 473/954] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:41.799419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:41.884567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616e4fe-cc60-42df-85e4-b9803d65f13a None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.884864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616e4fe-cc60-42df-85e4-b9803d65f13a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.885027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616e4fe-cc60-42df-85e4-b9803d65f13a None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.886946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 474/955] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:41.892392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.892675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.892945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.912427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf2929-4029-473f-a839-54c76aa6a484 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:41.914582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:41.917185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:41.951047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:41.951996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:41.952450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:41.952964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:41.953791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:41.954299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:41.956093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83391c21-9ab0-43ff-9baa-ee96b1a22e55 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:41.982385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 482/956] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:41.995468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:42.005913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 475/957] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:42.270468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615b66b-b6b1-4ba1-b8b8-567347b98589 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:42.270773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615b66b-b6b1-4ba1-b8b8-567347b98589 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:42.271222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615b66b-b6b1-4ba1-b8b8-567347b98589 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:42.272872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 483/958] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:42.274671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:42.278806 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:42.279191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:42.279561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:42.297745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:42.299425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:42.351583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:42.398023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 484/959] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:42.406483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7133011-fb7f-4770-ac05-7e85c3e9038c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:42.441853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:42.442506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:42.442779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:42.443045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:42.443621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:42.444973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:42.454191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:42.455460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:42.456586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 476/960] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:42.463368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:42.570882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013c50b-fdd7-41f5-b889-87d760e0cbb0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:42.607570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:42.608452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:42.608875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:42.609287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:42.610101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:42.610543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:42.612071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:42.617539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 485/961] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:42.623310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:42.683543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:42.688451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:42.736284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:42.748731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d226cc-4930-4974-97a7-3b2debf9bc6c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:42.787511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:42.788276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:42.788735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:42.789096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:42.790063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:42.790356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:42.792314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:42.798044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 477/962] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:42.816072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f9ce-f54c-47a3-bfca-cfca9cffb520 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:42.854447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:42.855423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:42.855915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:42.856401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:42.868845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:42.875618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:42.876630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:42.877472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:42.884466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 486/963] 10.222.0.50 () {56 vars in 1120 bytes} [Sat Sep 24 04:48:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:42.887308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:43.001628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a965-5e49-46a1-818f-80ccaa2c4f81 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:43.026629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:43.038118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:43.038711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.039431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.039699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.040362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:43.040657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:43.042145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:43.054084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 478/964] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:48:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:43.056513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:43.161682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:43.164696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395ac6b-774b-4478-8d1e-dff8a5ae16e8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:43.200177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:43.201217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.201716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.202223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.213123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:43.215395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:43.231520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:43.232485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:43.233377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:43.233377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:43.233377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:43.235466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 487/965] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:43.242039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:43.351578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cd203-aefc-4168-858a-ad0763e458e6 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:43.388075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:43.389044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.389521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.390036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.390926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:43.391425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:43.392981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:43.406266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 479/966] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:43.412239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:43.518619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2db39-f85e-4235-88a7-7362d0eeeeb6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:43.557371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:43.558323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.558748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.559186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.568147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:43.569597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-f6db43f9-bbf3-428f-a935-e65ce118c185 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:43.583410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:43.584343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 24 04:48:43.585265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:43.592604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 488/967] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:48:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:43.594269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:43.644591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:43.663881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee51873-a9fb-4f6a-987d-30c4cd54c37e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.664139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee51873-a9fb-4f6a-987d-30c4cd54c37e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.664367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee51873-a9fb-4f6a-987d-30c4cd54c37e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.666042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 489/968] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:43.670765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.671048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.671319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.692495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:43.694089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:43.702677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:43.731409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd8e4d-908e-4f9c-a5bf-b9864bf5e967 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:43.777646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:43.778303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 490/969] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:43.778438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.778602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.778877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.779611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:43.779914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:43.781331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:43.804759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 480/970] 10.222.0.50 () {56 vars in 1149 bytes} [Sat Sep 24 04:48:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:43.806269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:43.914062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4bbb5-80ba-4f13-8e06-895a0f52e93d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:43.949178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:43.949776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:43.950078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:43.950355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:43.962383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.020410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.021863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.088323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.128353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 491/971] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:43 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 04:48:44.135503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.187140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.228239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6224a9-fab5-4e52-b9ce-9e33be3cd809 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.235789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.273272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2156f85-da66-4593-b516-0e9370b6875d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.277642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.278615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.279092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.279576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.280377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.282382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-af77c4ef-1feb-4b26-9382-4c52c4fc2fdd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:44.292622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:44.293808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:44.294796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:44.297099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 481/972] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:44.298422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c287-408d-40d0-aa8e-7b993f7b2d81 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.298727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c287-408d-40d0-aa8e-7b993f7b2d81 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.298999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc8c287-408d-40d0-aa8e-7b993f7b2d81 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.300340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 482/973] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:44.303317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c287-408d-40d0-aa8e-7b993f7b2d81 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.313983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.314635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.314908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.315176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.325502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.326968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-cb0a0e5c-4208-4871-8417-5b6aec41e8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:44.344015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:44.345018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:44.345912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:44.345912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:44.345912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.345912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 492/974] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:44.347305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.347305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.347305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.365498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:44.366713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:44.418559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc8c287-408d-40d0-aa8e-7b993f7b2d81 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.446223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 493/975] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:44.446223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.461852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.464819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.466289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.467149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.467149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.467590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:44.468075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.470064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.481609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 483/976] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:44.482522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.524060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.546588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c50568-6d9f-41b8-bb51-cd88e835212c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.583985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.584954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.585436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.585896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.586830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:44.587321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.589056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.601678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 494/977] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:44.607044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.617667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3f3cd7-d2f3-4a44-ba46-2d360539423a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.665565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.667039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.667531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.668463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.669660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:44.670198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.672651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.685480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 484/978] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:44.687191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.716774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.736648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f091e-bbd4-4c38-9e4a-3af389b18a56 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.771662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.772293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.772570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.772843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.779635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.781070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:44.793507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:44.794441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:44.795365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.798170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:44.799423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 495/979] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:44.800147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.805507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923c30c-25ee-4039-8ee1-758791bc0e43 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.846462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.847072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.847352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.847625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.848459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] RBAC: Adding query filter params (interface=admin, service_id=f66195d944b045f3b5dee34e0c75b463, region_id=RegionOne) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:44.848882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.850329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.860583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 485/980] 10.222.0.50 () {56 vars in 1249 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/endpoints?service_id=f66195d944b045f3b5dee34e0c75b463&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:44.860583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.906599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1676e-637e-4582-821f-208766f013f1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.942379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:44.943219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:44.943648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:44.944103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:44.944887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:44.946489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:44.962600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.sql.core [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 04:48:44.962600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 04:48:44.962600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [parameters: {'user_id': '82a504bd2ca749f2b0e4ed8fcc138464', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 04:48:44.962600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81394) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 24 04:48:44.965234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcc976-ed3c-4fa7-86ef-9c413f2da664 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application conn.query(q) Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result.read() Sep 24 04:48:44.970808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 04:48:44.971683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application next(self.gen) Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application yield resource Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application next(self.gen) Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.session.rollback() Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.force_reraise() Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise self.value Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application session.commit() Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 24 04:48:44.972683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._prepare_impl() Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.session.flush() Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._flush(objects) Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application compat.raise_( Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application flush_context.execute() Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rec.execute(self) Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 24 04:48:44.973588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 24 04:48:44.974555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result = self._query(query) Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application conn.query(q) Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application result.read() Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 24 04:48:44.977557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [parameters: {'user_id': '82a504bd2ca749f2b0e4ed8fcc138464', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:44.981066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 24 04:48:44.983034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 496/981] 10.222.0.50 () {60 vars in 1131 bytes} [Sat Sep 24 04:48:44 2022] POST /identity/v3/users => generated 160 bytes in 172 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:44.985819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:45.008464 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:45.009098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.009394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.009656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.010960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:45.011273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:45.012762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:45.027081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 486/982] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:45.032884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:45.054951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:45.088880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5340e1-aaad-4909-9c20-3407f6a92fa3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:45.125403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:45.126191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.126477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.126751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.127360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:45.129177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:45.135214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290af81-edac-4a93-b609-704bf09ed12b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:45.138033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 497/983] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:44 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:48:45.138033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:45.181909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:45.182749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.183085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.183474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.191525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:45.193151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-fba10145-d2e0-40ae-ab5b-32b52c7d43fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:45.206896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:45.207933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:45.208903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 487/984] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:45.210196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00241-fadd-4ee6-89e4-3f98b18ff371 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.210196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00241-fadd-4ee6-89e4-3f98b18ff371 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.210196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00241-fadd-4ee6-89e4-3f98b18ff371 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.214319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 488/985] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:45.215792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00241-fadd-4ee6-89e4-3f98b18ff371 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:45.230180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:45.266896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee043f-63c5-4809-b47d-42334ab1d20d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:45.305798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:45.306100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:45.306992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.307102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.307895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.308626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] RBAC: Adding query filter params (role.id=3e2c1a2d80234e7fbf13a69c075de868, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=8b1bc0d6bb8248cd86c8ac462709e326) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:45.308962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:45.310753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8547ae82-06c7-48f4-a527-58176b2f9f2e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:45.318112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 498/986] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:45 2022] GET /identity/v3/role_assignments?user.id=8b1bc0d6bb8248cd86c8ac462709e326&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=3e2c1a2d80234e7fbf13a69c075de868 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:45.319506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7149-e3b0-4b3f-a26c-82fce0f378f4 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.319963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7149-e3b0-4b3f-a26c-82fce0f378f4 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.320249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb7149-e3b0-4b3f-a26c-82fce0f378f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.331486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00241-fadd-4ee6-89e4-3f98b18ff371 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:45.341974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-21eb7149-e3b0-4b3f-a26c-82fce0f378f4 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:45.343509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-21eb7149-e3b0-4b3f-a26c-82fce0f378f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:45.375529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:45.376151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.376551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.376705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.378282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:45.378282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:45.381115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:45.422345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 489/987] 10.222.0.50 () {56 vars in 1101 bytes} [Sat Sep 24 04:48:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:45.428768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 499/988] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:45.699538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:45.800576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce6910-f508-47d1-a951-0a57672294ec None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:45.834592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e442ba-9f4b-45dd-94e5-5fd66b5c7bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:45.835196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e442ba-9f4b-45dd-94e5-5fd66b5c7bd8 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:45.835472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e442ba-9f4b-45dd-94e5-5fd66b5c7bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:45.835751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e442ba-9f4b-45dd-94e5-5fd66b5c7bd8 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:45.836407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e442ba-9f4b-45dd-94e5-5fd66b5c7bd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:45.837759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e442ba-9f4b-45dd-94e5-5fd66b5c7bd8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:45.848342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:45.862385 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 490/989] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:45 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:47.376971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e4f87-9017-4f3a-956c-c12fcd72eabf None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:47.377521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e4f87-9017-4f3a-956c-c12fcd72eabf None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:47.377521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e4f87-9017-4f3a-956c-c12fcd72eabf None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:47.379270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 500/990] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:47.384235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e445af7-00fa-44ae-ad9f-0a92e7bcb512 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:47.384610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e445af7-00fa-44ae-ad9f-0a92e7bcb512 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:47.384774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e445af7-00fa-44ae-ad9f-0a92e7bcb512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:47.404792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6e445af7-00fa-44ae-ad9f-0a92e7bcb512 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:47.406060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6e445af7-00fa-44ae-ad9f-0a92e7bcb512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:47.482059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 491/991] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:47.521758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c11be3-0074-4428-bb63-b1e89fe013cb None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:47.522108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c11be3-0074-4428-bb63-b1e89fe013cb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:47.522330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c11be3-0074-4428-bb63-b1e89fe013cb None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:47.523787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 501/992] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:47.529374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:47.529663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:47.529933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:47.547856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:47.549025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:47.617768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 492/993] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:47.750217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c11be3-0074-4428-bb63-b1e89fe013cb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:47.813296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:47.857613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c11be3-0074-4428-bb63-b1e89fe013cb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:47.884212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:47.891651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:47.892286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:47.892561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:47.892830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:47.893410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:47.894791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:47.904020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:47.905170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:47.906607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 502/994] 10.222.0.50 () {56 vars in 1126 bytes} [Sat Sep 24 04:48:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:47.911281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:47.955473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.004173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c656de-f7a7-4f78-baad-ebae942902d5 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.031759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab2ba2-f6dc-46aa-9baf-594a52edbe84 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.039913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.040538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.041105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.041105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] PATH_INFO: `/v3/services/image` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.041662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.043055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-94faa39d-bd65-4fc7-a6d4-a469ed5a8d3c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:48.051502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:48.052691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:48.053701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:48.057453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 493/995] 10.222.0.50 () {56 vars in 1105 bytes} [Sat Sep 24 04:48:47 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:48.057869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.058859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3a76-7339-4287-bff8-47323c4428ea None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.059151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3a76-7339-4287-bff8-47323c4428ea None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.059367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3a76-7339-4287-bff8-47323c4428ea None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.060612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 494/996] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:48.061343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3a76-7339-4287-bff8-47323c4428ea None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.067399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.068002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.068277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.068548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.069168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:48.069464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.070857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d5d49-e3c7-432b-8f10-3cd6609591bc None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.078158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 503/997] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:48.078158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.078158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.078158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.096019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:48.097189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:48.162831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 504/998] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:48.163186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3a76-7339-4287-bff8-47323c4428ea None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.163474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.195484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.214064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.214755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.215100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.215445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.216132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] RBAC: Adding query filter params (name=image) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:48.216507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.217949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.226160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 495/999] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:48.228223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.267522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7d448-2a9e-42d0-a472-fc8b26757472 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.291177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.303433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.304049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.304324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.304590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.305209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:48.305506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.306886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.312347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 505/1000] 10.222.0.50 () {56 vars in 1122 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:48.318065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.347929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e23f-8522-4fb5-b0df-a59bc50c388b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.388159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.388804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.389015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.389284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.389901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] RBAC: Adding query filter params (type=image) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:48.390225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.391572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.395920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 496/1001] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:48.428644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2b36f-dde6-4f04-a86f-9d2247eab839 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.445509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.467600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.468219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.468497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.468769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.469445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.470890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.490194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 506/1002] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:48.502555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.569267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9782e-8869-4c57-93ce-41056a97e093 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.614900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.615544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.615816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.616140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.616726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.618141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:48.628651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:48.629769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:48.630679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 497/1003] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:48.636114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.694878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.752201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9f520-a27b-4141-808f-127dd610318e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.787462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.788070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.788343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.788609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.789227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:48.789520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.790902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.800370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 507/1004] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:48.801841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:48.902221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e515852-5179-4880-9b65-62ab95b75bf4 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:48.909872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:48.937752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:48.938380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:48.938654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:48.938919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:48.939537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:48.939829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:48.941175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:48.950873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 498/1005] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:48.952396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:49.054386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c202b89-c5bd-40b2-90e4-d71e0f96dfe3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:49.065204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:49.089706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:49.090343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:49.090621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:49.090892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:49.091585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] RBAC: Adding query filter params (interface=public, service_id=e3e21ecd5d8049389019f8b315114ea9, region_id=RegionOne) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:49.091888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:49.093236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b0dcd-6622-4c72-bfef-bbf927d08ea1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:49.102783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 508/1006] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:48 2022] GET /identity/v3/endpoints?service_id=e3e21ecd5d8049389019f8b315114ea9&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:49.104066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:49.205706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944d187-6c6e-49a2-b076-cc90838df89b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:49.241634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c478dac-8f40-4b26-a6cb-ec92b711ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:49.242278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c478dac-8f40-4b26-a6cb-ec92b711ae9f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:49.242557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c478dac-8f40-4b26-a6cb-ec92b711ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:49.242826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c478dac-8f40-4b26-a6cb-ec92b711ae9f None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:49.243423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c478dac-8f40-4b26-a6cb-ec92b711ae9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:49.244827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c478dac-8f40-4b26-a6cb-ec92b711ae9f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:49.258025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 499/1007] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:49 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:48:50.605870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0657e-8997-4e46-a710-8a7726b6a58e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:50.606450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0657e-8997-4e46-a710-8a7726b6a58e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:50.606450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0657e-8997-4e46-a710-8a7726b6a58e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:50.607831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 509/1008] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:50.613071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:50.613488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:50.613648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:50.632909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:50.634110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:50.669243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:50.688895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468698eb-a940-4473-b98f-419ed3b8bdfe None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:50.689190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468698eb-a940-4473-b98f-419ed3b8bdfe None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:50.689466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468698eb-a940-4473-b98f-419ed3b8bdfe None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:50.691054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 510/1009] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:50.695844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33052685-013a-4cee-8852-87f16774df5a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:50.696130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33052685-013a-4cee-8852-87f16774df5a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:50.696406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33052685-013a-4cee-8852-87f16774df5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:50.718024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-33052685-013a-4cee-8852-87f16774df5a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:50.722060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 500/1010] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:50.722193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-33052685-013a-4cee-8852-87f16774df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:50.786038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 511/1011] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:50.987379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:51.055718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33052685-013a-4cee-8852-87f16774df5a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:51.090292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586d205-be7a-488f-b4f4-cba3477116db None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:51.127205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:51.127818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:51.128093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:51.128629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:51.129288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:51.130686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:51.146705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 501/1012] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:50 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:51.164883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33052685-013a-4cee-8852-87f16774df5a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:51.169479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:51.213329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fe3d1-1388-4453-8cde-e6c4bfc66244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:51.214529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fe3d1-1388-4453-8cde-e6c4bfc66244 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:51.214807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fe3d1-1388-4453-8cde-e6c4bfc66244 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:51.215085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fe3d1-1388-4453-8cde-e6c4bfc66244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:51.216250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fe3d1-1388-4453-8cde-e6c4bfc66244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:51.217854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fe3d1-1388-4453-8cde-e6c4bfc66244 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:51.219780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 512/1013] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:51.225790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:51.255606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:51.348777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053623b4-b8c0-436d-9385-dc6cb735b389 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:51.383151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fc77a-c34f-4652-bd08-ff58d114a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:51.383770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fc77a-c34f-4652-bd08-ff58d114a808 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:51.384048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fc77a-c34f-4652-bd08-ff58d114a808 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:51.384319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fc77a-c34f-4652-bd08-ff58d114a808 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:51.384977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fc77a-c34f-4652-bd08-ff58d114a808 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:51.386465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fc77a-c34f-4652-bd08-ff58d114a808 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:51.422047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 502/1014] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:48:51 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:51.432125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2e0cd-5f59-4137-bec1-a4a25d60536f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:51.433391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2e0cd-5f59-4137-bec1-a4a25d60536f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:51.433678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2e0cd-5f59-4137-bec1-a4a25d60536f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:51.435327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 513/1015] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:51.441084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:51.441414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:51.441641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:51.461201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:51.462443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:51.536825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 503/1016] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:51 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:51.805698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2e0cd-5f59-4137-bec1-a4a25d60536f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:51.909062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2e0cd-5f59-4137-bec1-a4a25d60536f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:51.946722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:51.947356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:51.947638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:51.947912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:51.949103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:51.950521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:51.959472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:51.960444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:51.961315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 514/1017] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:51.967978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:51.976178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:52.075648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94abc8-d603-4ee2-85fd-87b5fbda7c4f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:52.109827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:52.110465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:52.110748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:52.111024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:52.111657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:52.111960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:52.113324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:52.123504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 504/1018] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:52.124451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:52.156510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:52.227116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e73677-5272-4cd6-8683-9936a16630d4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:52.264007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:52.264641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:52.264939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:52.265220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:52.265892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:52.266262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:52.267604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c45e90-e141-4cdf-9015-38e113af902e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:52.272517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 515/1019] 10.222.0.50 () {56 vars in 1112 bytes} [Sat Sep 24 04:48:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:52.277982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:52.354990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:52.387628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d21fcf-f211-47b6-9207-951ce8d59521 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:52.421612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22727607-68b9-4fda-ad4e-f0d6a165abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:52.422235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727607-68b9-4fda-ad4e-f0d6a165abd7 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:52.422527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727607-68b9-4fda-ad4e-f0d6a165abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:52.422799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727607-68b9-4fda-ad4e-f0d6a165abd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:52.423468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22727607-68b9-4fda-ad4e-f0d6a165abd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:52.424859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22727607-68b9-4fda-ad4e-f0d6a165abd7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:52.445114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 505/1020] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:48:53.334808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdc5db-392a-4e1a-ae13-d297cbea9190 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:53.335372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdc5db-392a-4e1a-ae13-d297cbea9190 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:53.335372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdc5db-392a-4e1a-ae13-d297cbea9190 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:53.336878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 516/1021] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:53.342105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca851b-8cd0-4da6-b50d-5b26680b6edb None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:53.342538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca851b-8cd0-4da6-b50d-5b26680b6edb None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:53.342708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca851b-8cd0-4da6-b50d-5b26680b6edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:53.360757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-01ca851b-8cd0-4da6-b50d-5b26680b6edb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:53.361949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-01ca851b-8cd0-4da6-b50d-5b26680b6edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:53.436776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 506/1022] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:53.628175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04270-c798-4569-818e-160f603942eb None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:53.628704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04270-c798-4569-818e-160f603942eb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:53.628905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e04270-c798-4569-818e-160f603942eb None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:53.630433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 517/1023] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:53.635365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:53.636208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:53.636473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:53.654330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:53.655506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:53.702108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04270-c798-4569-818e-160f603942eb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:53.703097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:53.725246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 507/1024] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:53.800335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:53.807997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e04270-c798-4569-818e-160f603942eb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:53.841748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:53.842395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:53.842670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:53.842936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] PATH_INFO: `/v3/services/image` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:53.843507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:53.844850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:53.853490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:53.854578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:53.855429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:53.858326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 518/1025] 10.222.0.50 () {56 vars in 1105 bytes} [Sat Sep 24 04:48:53 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:53.860439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:53.958347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0bae-b44d-468e-9d91-809fc1a8e95d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:53.991777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:53.992379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:53.992654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:53.992920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:53.993537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] RBAC: Adding query filter params (name=image) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:53.993828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:53.995326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:54.000193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 508/1026] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:54.005916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:54.007070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:54.056527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:54.110281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757294-81c1-42e4-a1c1-9b49095e9e53 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:54.115909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232922-e505-4a77-beea-3da4798ca0ac None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:54.144900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:54.145519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.145796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.146089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.147184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:54.148588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:54.150368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 509/1027] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:54.152053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:54.152663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.152944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.153218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.153847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:54.154178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:54.155526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:54.156114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:54.165319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 519/1028] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:54.166351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:54.193344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:54.258783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f798a6-8acf-4a18-adc4-562a5fe57fb4 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:54.273390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:54.278628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136547a8-0da9-4738-9730-71ed1f7fdeb2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:54.294442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:54.295045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.295315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.295584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.296234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:54.297653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:54.320233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 510/1029] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:48:54 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:54.320660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ed131-cbff-4725-9505-b818653928de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:54.321277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ed131-cbff-4725-9505-b818653928de None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.321545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ed131-cbff-4725-9505-b818653928de None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.321861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ed131-cbff-4725-9505-b818653928de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.322635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ed131-cbff-4725-9505-b818653928de None admin] RBAC: Adding query filter params (interface=public, service_id=8d5142f7a5324f6aa50c0c7f27f1e1bb, region_id=RegionOne) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:54.322948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ed131-cbff-4725-9505-b818653928de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:54.324622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ed131-cbff-4725-9505-b818653928de None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:54.340932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 520/1030] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:48:54 2022] GET /identity/v3/endpoints?service_id=8d5142f7a5324f6aa50c0c7f27f1e1bb&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:54.342428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:54.454430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642aa3b-13f5-4a5d-8a3b-219d60f11eed None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:54.490530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8d0d2-29cc-45f3-ae98-2e38444c8a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:54.491153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8d0d2-29cc-45f3-ae98-2e38444c8a8c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.491433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8d0d2-29cc-45f3-ae98-2e38444c8a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.491689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8d0d2-29cc-45f3-ae98-2e38444c8a8c None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.492300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8d0d2-29cc-45f3-ae98-2e38444c8a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:54.493662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb8d0d2-29cc-45f3-ae98-2e38444c8a8c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:54.510111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 511/1031] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:48:54 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:48:54.834317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5313fc1-108f-4572-b59e-f7e51bd23703 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.834521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5313fc1-108f-4572-b59e-f7e51bd23703 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.834798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5313fc1-108f-4572-b59e-f7e51bd23703 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.836161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 521/1032] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:54.841926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:54.842251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:54.842528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:54.860735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:54.861930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:54.889828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:54.930828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 512/1033] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:55.197402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5313fc1-108f-4572-b59e-f7e51bd23703 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:55.287174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:55.301123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5313fc1-108f-4572-b59e-f7e51bd23703 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:55.337824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:55.338482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:55.339394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:55.339702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:55.349901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:55.354746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:55.355676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:55.356520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 522/1034] 10.222.0.50 () {56 vars in 1120 bytes} [Sat Sep 24 04:48:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:55.361815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:55.464974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9a40b2-0ae3-4f71-81bd-0e9033ae97aa None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:55.497791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:55.498419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:55.498700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:55.498979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:55.499638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:55.499938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:55.501294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:55.514669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 513/1035] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:48:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:55.516049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:55.616813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e5c2b-fa08-4425-84d8-17862cfd918a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:55.656377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:55.657016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:55.657356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:55.657632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:55.668306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:55.670034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:55.678764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:55.687184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:55.688209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:55.689103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:55.689103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:55.689103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:55.695455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 523/1036] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:55.696463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:55.725853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:55.801893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455537b0-fb8f-4e2c-b5ae-a4de28022003 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:55.837280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:55.838214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:55.839134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:55.839580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:55.840667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:55.841166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:55.843523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:55.858356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 514/1037] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:55.866121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:55.965109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b72cae-a3f0-422e-a9c9-c0e9bde66dff None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:55.997811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:55.998390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:55.998664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:55.998934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.006548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:56.007964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:48:56.019420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:56.020417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:48:56.021370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:56.026269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:56.027057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 524/1038] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:48:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:56.110865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:56.127860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ea820-a896-4a24-8b80-3666aaafb123 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:56.162731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:56.163380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.163650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.163962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.164685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:56.164984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:56.166425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:56.189466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 515/1039] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:48:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:56.190747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:56.252182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:56.294376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91490312-ec77-4abf-b0d0-00f782b39ea1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:56.327608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e54a2-1b55-4e28-884d-5ffdcfc5de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:56.328219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e54a2-1b55-4e28-884d-5ffdcfc5de2f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.328497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e54a2-1b55-4e28-884d-5ffdcfc5de2f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.328769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e54a2-1b55-4e28-884d-5ffdcfc5de2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.397099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e54a2-1b55-4e28-884d-5ffdcfc5de2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:56.398599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1e54a2-1b55-4e28-884d-5ffdcfc5de2f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:56.455257 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:56.505628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 525/1040] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:56 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:48:56.507223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:56.602550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f859a-624c-4fac-ae74-e60f5c1af3b8 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.602913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f859a-624c-4fac-ae74-e60f5c1af3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.603117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f859a-624c-4fac-ae74-e60f5c1af3b8 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.604718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 526/1041] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:48:56.610570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598f69e-d409-4185-8f43-6b3fb06b8cc0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.610917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598f69e-d409-4185-8f43-6b3fb06b8cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.611084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598f69e-d409-4185-8f43-6b3fb06b8cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.613472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e87896-b12d-4fbb-9c6a-d5fac663d3ab None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:56.629780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a598f69e-d409-4185-8f43-6b3fb06b8cc0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:56.630993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a598f69e-d409-4185-8f43-6b3fb06b8cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:56.658051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:56.658690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.658968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.659239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.669840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:56.671298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:56.673837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:48:56.687578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:56.688640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:48:56.689527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:48:56.689527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:48:56.689527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:56.689527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 516/1042] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:56.694127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:56.715423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 527/1043] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:56.716739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15895f-f3df-468c-ab67-8f8ec42d0a72 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.717176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15895f-f3df-468c-ab67-8f8ec42d0a72 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.717358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15895f-f3df-468c-ab67-8f8ec42d0a72 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.718885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 528/1044] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:56.723429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406e573-e67e-4485-b6cb-952696865405 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.723707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406e573-e67e-4485-b6cb-952696865405 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.723978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406e573-e67e-4485-b6cb-952696865405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.742281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f406e573-e67e-4485-b6cb-952696865405 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:56.743702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f406e573-e67e-4485-b6cb-952696865405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:56.805569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf4af3-b5d3-4eb1-b28a-a0990278b69b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:56.818055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 529/1045] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:48:56.843633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:56.844235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:56.844509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:56.844775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:56.845412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:56.845711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:56.847110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:56.863736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 517/1046] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:56.864929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406e573-e67e-4485-b6cb-952696865405 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:56.887296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:56.966133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406e573-e67e-4485-b6cb-952696865405 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:56.986060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.001339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.001971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.002272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.002539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.009777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.011267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:48:57.022272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:57.028265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 530/1047] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:48:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:57.029223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.051720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.096345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d6eaf-9de5-40b7-9981-e8ad116f3d62 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.111290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.129850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.130505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.130790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.131071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.131353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eda83-5cd3-4b56-aaa1-4a1b0a3e5df2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.142098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:48:57.146803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:57.147721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:57.148582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 518/1048] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:48:57.149282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.166829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.167436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.167713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.167983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.168667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:57.168968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.170427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.182078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 531/1049] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:57.183365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.246665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3f2dc-6626-47a5-ae71-6f18be87470f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.263132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.280403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664089a-2b22-42fa-96c8-77232acbeee5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.280986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.281608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.281888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.282192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] PATH_INFO: `/v3/services/image` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.282762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.284120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:48:57.292746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:48:57.293797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:48:57.294664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 519/1050] 10.222.0.50 () {56 vars in 1105 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/services/image => generated 86 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:48:57.295442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.313797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.314427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.314710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.314977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.315633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:57.315929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.317606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.325258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 532/1051] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:57.326044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.328439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.399909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa9a30-e91e-416b-b179-8ef6264d6fba None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.426544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182645ef-a3cc-4fed-92ed-e6778d1c2df7 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.431902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.433040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.433326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.433601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.434351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] RBAC: Adding query filter params (role.id=3e2c1a2d80234e7fbf13a69c075de868, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=2451d53fb9974222827b6520b8bce5f3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:57.434654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.436086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.441580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 520/1052] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/role_assignments?user.id=2451d53fb9974222827b6520b8bce5f3&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=3e2c1a2d80234e7fbf13a69c075de868 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:57.442296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.477271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.477901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.478208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.478482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.479173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:57.479948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.481346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.491691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 533/1053] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:57.492735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.500986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.561931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e10c9-6bc7-426a-9b33-3cb42e9c9d1d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.566167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.600141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.600787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.601069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.601344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] PATH_INFO: `/v3/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.609437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29b609-9dae-4927-a4ca-bb96ef8d63a4 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.616644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] RBAC: Authorizing `identity:get_user(user_id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.618145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.620340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 521/1054] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd => generated 285 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:57.626625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.649385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.649797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.650175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.650391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.651023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:57.651380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.652865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.658254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 534/1055] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:57.663617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:57.741654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeac52f-6c7e-4073-b5de-5cfdf3209d2f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.743460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.779472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d80ae-a984-43a2-89b6-f541dd3b54d8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:57.780541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.781401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.781845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.782209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.810497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:57.817627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:57.818274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:57.818556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:57.818834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:57.819508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.820874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.850034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 535/1056] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:48:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:48:57.863990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:57.865614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:57.979390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 522/1057] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 04:48:57.985836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:58.100509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357eeed-b8d3-4c01-b1e1-9f72b33db67a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:58.134908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd9ba7-5edf-457d-aba3-7e3d89a22a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:58.136154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd9ba7-5edf-457d-aba3-7e3d89a22a75 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:58.136443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd9ba7-5edf-457d-aba3-7e3d89a22a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:58.136714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd9ba7-5edf-457d-aba3-7e3d89a22a75 None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:58.137841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd9ba7-5edf-457d-aba3-7e3d89a22a75 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca87a9a417a4d168187240c172b69a0)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:58.139300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd9ba7-5edf-457d-aba3-7e3d89a22a75 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:58.141105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 536/1058] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:48:57 2022] GET /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:48:58.147322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:48:58.149523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:58.254848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a80e43-381a-4e68-be4a-60f3637e577b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:48:58.288214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:48:58.289107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:58.289592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:58.290085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:58.290991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=6ca87a9a417a4d168187240c172b69a0, user.id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:48:58.291503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:48:58.293675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666adc91-a3f3-423f-87aa-06e717fc5ee8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:48:58.304082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 523/1059] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:48:58 2022] GET /identity/v3/role_assignments?user.id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd&scope.project.id=6ca87a9a417a4d168187240c172b69a0&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:48:59.595526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248515ce-b685-4c5c-815e-f5e0293581b3 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:59.596081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248515ce-b685-4c5c-815e-f5e0293581b3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:59.596081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248515ce-b685-4c5c-815e-f5e0293581b3 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:59.597434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 537/1060] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:48:59.602729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d3b6-3fd9-4a09-a873-574e4f3be53f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:48:59.603143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d3b6-3fd9-4a09-a873-574e4f3be53f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:48:59.603284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d3b6-3fd9-4a09-a873-574e4f3be53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:48:59.622122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-be29d3b6-3fd9-4a09-a873-574e4f3be53f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:48:59.623292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-be29d3b6-3fd9-4a09-a873-574e4f3be53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:48:59.689957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:48:59.698196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 524/1061] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:48:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:48:59.969192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248515ce-b685-4c5c-815e-f5e0293581b3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.008172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4d9c-ffdc-472a-855f-bc3803ccb4a1 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.009258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4d9c-ffdc-472a-855f-bc3803ccb4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.009553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f4d9c-ffdc-472a-855f-bc3803ccb4a1 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.011242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 525/1062] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:00.015992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.016690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.016971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.024347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.039757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:00.041335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:00.077800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248515ce-b685-4c5c-815e-f5e0293581b3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.114064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.114325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.114572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.116525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.119541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 526/1063] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:00.130687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:00.135563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:00.136573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:49:00.137411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:00.141762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 538/1064] 10.222.0.50 () {56 vars in 1120 bytes} [Sat Sep 24 04:48:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:00.142764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.240526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed28741c-a4dc-4ea9-881a-f6dcfa32efae None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.266174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.275776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.276895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.277169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.277435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.278116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:00.278415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:00.279775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:00.292651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 527/1065] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:00.293907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.379100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.415827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7cdd-f259-47c0-a544-09b9d10f9b12 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.430268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.456198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.456827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.457105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.457372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.467587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:00.469017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:00.483523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b1fc1-8573-432c-a26c-f7f1db5d4f51 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b94948fb-0b1c-4cd8-98ba-2225f4e0b63c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:49:00.484019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:00.484942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:49:00.486572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:49:00.486572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:00.486572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:00.486572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 539/1066] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:00.486572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a304-5831-4a8c-ac99-3b588b4a4a0d None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.486572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a304-5831-4a8c-ac99-3b588b4a4a0d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.486924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a304-5831-4a8c-ac99-3b588b4a4a0d None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.488215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 540/1067] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:00.490666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a304-5831-4a8c-ac99-3b588b4a4a0d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.520383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.520983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.521280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.521535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.575887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.588136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:00.589599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:00.591266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a304-5831-4a8c-ac99-3b588b4a4a0d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.625373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.625986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.626273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.626550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.627188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:00.627512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:00.629120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ff61c-a167-4678-b1fc-898d89ca94e6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:00.640139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 541/1068] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:00.640139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.640139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.640139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.660684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:00.661930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:00.678676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.690316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 528/1069] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 04:49:00.691138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.731363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 542/1070] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:00.732196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.788825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.802806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2bd66-66f3-4f44-81c3-a155e7908a8c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.840649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19ffef-f331-414a-976a-756dbbe6328c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:00.843793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.844426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.844701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.845028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.852380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:00.854127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:49:00.868352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:00.869461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:00.870583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 529/1071] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:00.875236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.877853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:00.878520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:00.879056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:00.879331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:00.889774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:00.891481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:49:00.907692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:00.908670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:49:00.909596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:49:00.909596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:00.909596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:00.909596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 543/1072] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:00.914884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:00.918545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:00.979021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f5d0b-e2d1-4bda-83cb-82f1f1093ab2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.013347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.013955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.014260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.014531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.015257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:01.015557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.016976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.021388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d118f8-29e3-49fa-9f0b-43a862b8f024 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.033539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 530/1073] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:01.034358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.043971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.055009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.055618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.055892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.056160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.056814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:01.057097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.058968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.070169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 544/1074] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:01.070998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.143042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.151594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37259c1d-fb1f-4ebe-a5f1-b1f2475f23d7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.188343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be30efaf-8903-4504-a416-366e1cb7e801 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.189010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.189657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.189933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.190337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.202322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:01.207410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:01.208473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:01.209311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 531/1075] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:01.215052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.233989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.234667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.234949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.235221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.247136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.248589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:49:01.264027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:01.265143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:49:01.266404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:49:01.266404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:01.266404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:01.266404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 545/1076] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:01.269801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.299677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.341467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58022453-f6fc-4900-bedc-f0f5faa33b0b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.385055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.386198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9a2eb-b1d4-45dc-a137-99a216bdc1e0 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.386325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.386506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.386749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.387421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:01.387721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.389093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.397780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 532/1077] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:01.403076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.403931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.422137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.422750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.423029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.423301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.423937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:01.424239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.425590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.438041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 546/1078] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:01.443509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.503487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533fee0-9c96-4b7a-921e-fbb3ec61b217 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.529199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.537667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.538333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.538618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.538928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] PATH_INFO: `/v3/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.540082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] RBAC: Authorizing `identity:get_user(user_id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.541545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.543649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 533/1079] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:01.546113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105398c8-2724-4b74-a519-942c10dcbd91 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.549356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.580475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.581017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.581295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.581568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.588404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.590790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:49:01.602191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:01.603138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:01.604042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 547/1080] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:49:01.609024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.644206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.656045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a7513-ac0a-46a1-b617-bdd06518f610 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.690466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.691081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.691361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.691636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.692747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca87a9a417a4d168187240c172b69a0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.694213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.695839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 534/1081] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:01.702430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.714157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf3fa4-cdda-41cc-a319-43deb50ac127 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.748698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.749300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.749579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.749851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.750576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:01.750903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.752324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.763138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.764512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 548/1082] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:49:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:01.770208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:01.801418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257944bb-e604-46fd-b3fa-8f1a4ecf0a18 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.833630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaac09-9682-4ded-a525-c185727310ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.834264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaac09-9682-4ded-a525-c185727310ec None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.834540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaac09-9682-4ded-a525-c185727310ec None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.834815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaac09-9682-4ded-a525-c185727310ec None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.836728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaac09-9682-4ded-a525-c185727310ec None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca87a9a417a4d168187240c172b69a0, user_id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.838169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaac09-9682-4ded-a525-c185727310ec None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.851263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 535/1083] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:49:01 2022] PUT /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 04:49:01.868529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:01.876192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be57d06-95e2-4e35-bac4-091c8ed5bd4c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:01.912880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fede2fc-77e9-4082-9f56-62788980c182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:01.913566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede2fc-77e9-4082-9f56-62788980c182 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:01.913845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede2fc-77e9-4082-9f56-62788980c182 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:01.914158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fede2fc-77e9-4082-9f56-62788980c182 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/2451d53fb9974222827b6520b8bce5f3/roles/3e2c1a2d80234e7fbf13a69c075de868` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:01.916375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fede2fc-77e9-4082-9f56-62788980c182 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=2451d53fb9974222827b6520b8bce5f3, role_id=3e2c1a2d80234e7fbf13a69c075de868)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:01.917891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fede2fc-77e9-4082-9f56-62788980c182 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:01.931910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 549/1084] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:49:01 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/2451d53fb9974222827b6520b8bce5f3/roles/3e2c1a2d80234e7fbf13a69c075de868 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 04:49:03.976013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4849-b076-4cbe-a302-c69f9b1db22b None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:03.976594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4849-b076-4cbe-a302-c69f9b1db22b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:03.976703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4849-b076-4cbe-a302-c69f9b1db22b None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:03.978090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 536/1085] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:03.984067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55942637-950a-4c0e-812b-fd9673448a24 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:03.984621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55942637-950a-4c0e-812b-fd9673448a24 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:03.984721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55942637-950a-4c0e-812b-fd9673448a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.003232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-55942637-950a-4c0e-812b-fd9673448a24 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:04.004657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-55942637-950a-4c0e-812b-fd9673448a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:04.068539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 550/1086] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:04.079443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15669ce8-5d5d-4b1c-b382-3e2705895421 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.079734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15669ce8-5d5d-4b1c-b382-3e2705895421 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.080013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15669ce8-5d5d-4b1c-b382-3e2705895421 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.081328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 537/1087] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:04.086547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.086834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.087109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.104648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:04.105834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:04.169795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 551/1088] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:04.339993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15669ce8-5d5d-4b1c-b382-3e2705895421 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.440285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15669ce8-5d5d-4b1c-b382-3e2705895421 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.440715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.457564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:04.464293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:04.476182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.476809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.477088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.477360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.488447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:04.493260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:04.494370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:04.495969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 538/1089] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:04.503534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.541699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c422b-9a8c-4a01-bccd-0831c6f1cb4f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.575469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.576093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.576371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.576643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.586528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:04.591564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:04.592504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:04.593343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 552/1090] 10.222.0.50 () {56 vars in 1120 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:49:04.598544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.602201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a2e12-da46-4185-b2af-b55ead843cc3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.635824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.636422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.636702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.637030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.637695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:04.638019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:04.639380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:04.648062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 539/1091] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:04.654949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.678135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:04.700488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:04.701088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd90974-b671-439a-9534-90fd7a98fd0e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.736759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.737358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.737638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.737901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.738596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:04.738894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:04.740238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:04.748376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 553/1092] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:04.753598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faba62d-cef9-44f7-b1c4-0f93a5b116e7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.754376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.788388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.789013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.789291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.789562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] PATH_INFO: `/v3/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.790710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] RBAC: Authorizing `identity:get_user(user_id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:04.792156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:04.793702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 540/1093] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/users/9d1bf7d1cc81439bbb5d4fc9ffb3ddcd => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:04.799365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.856360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9ecf5-4a7f-4052-bde3-8a84af07a76f None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.889191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.889819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.890125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.890401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:04.897351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:04.900635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c8f0d-a64b-4abe-99c9-12c38e3afac0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:04.901784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:04.903396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:04.915869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:49:04.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:04.918872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:49:04.919765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:49:04.919765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:04.919765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:04.919765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 554/1094] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:04.924692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:04.943387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:04.943992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:04.944273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:04.944553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.010087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.012050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.021426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d332abe-2059-499f-bc74-5b1d3fdf7293 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.054838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.055443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.055719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.055991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.056624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:05.056919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.058883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.070389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 555/1095] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:05.075908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.110041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 541/1096] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 04:49:05.115842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.129202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.129752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.179480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561c76-facf-452c-80fa-2e916ff9efa2 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.214914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.215643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.215777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe06a8-d83b-4412-8626-b3a4a6a734af None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.215921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.216187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.223128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.224551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:49:05.235829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:05.236872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:05.237824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 556/1097] 10.222.0.50 () {56 vars in 1093 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:49:05.242549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.248906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.249539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.249820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.250126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.251242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca87a9a417a4d168187240c172b69a0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.252666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.259477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 542/1098] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:05.260561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.338291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.341249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e996a6-243f-47ae-8fb3-62ce279687a3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.351680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.357861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e4a7-a9df-43f6-9c2c-630ab5f862b3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.375760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.376293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.376567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.376836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.377548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:05.377850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.379552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.393413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.394033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.394774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.395066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.395795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=6ca87a9a417a4d168187240c172b69a0, user.id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:05.396105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.397513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.397622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 557/1099] 10.222.0.50 () {56 vars in 1133 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:05.403274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.409373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 543/1100] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/role_assignments?user.id=9d1bf7d1cc81439bbb5d4fc9ffb3ddcd&scope.project.id=6ca87a9a417a4d168187240c172b69a0&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:05.512243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7ca92-46c4-4314-9b3a-0fc6fc09b930 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.548201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.548824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.549094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.549368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.584035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.617833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.619311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.721044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 558/1101] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:05.726739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.823292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84dc45-82ee-46bf-a9e3-fd3f20b5b217 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:05.855430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:05.856047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:05.856320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:05.856585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:05.866821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:05.868240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:05.874424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:49:05.883068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:05.884134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:49:05.886140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:49:05.886140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:49:05.886140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:05.886140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 544/1102] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:05.890342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:05.954956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:05.987094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8c900-9217-4bc6-99a3-9c3b5cb2f26d None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:06.019753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:06.020350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:06.020625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:06.020891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:06.021511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:06.022072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:06.023172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:06.033935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 559/1103] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:06.039313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:06.138201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabf2d9-a59e-4fe1-b2b5-b12dd2c65292 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:06.170207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:06.170822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:06.171096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:06.171363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:06.177707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:06.179161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:49:06.190686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:06.191680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:06.192577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 545/1104] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:49:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:06.197544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:06.292984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883961b-899c-4e90-87b9-9ff838dfce65 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:06.320165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:06.328328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:06.328936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:06.329241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:06.329514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:06.330264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:06.330568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:06.332007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6920b-ceaa-478a-9f61-dfbb7803fc4f None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:06.344071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 560/1105] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:49:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:06.350100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:06.407509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:06.448158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7d5c6-8065-469b-abb6-7a03cdd0cc5c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:06.480818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:06.481489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:06.481771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:06.482069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:06.482784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] RBAC: Adding query filter params (role.id=3e2c1a2d80234e7fbf13a69c075de868, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=2451d53fb9974222827b6520b8bce5f3) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:06.483088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:06.484504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29849790-f0e4-4855-9492-ffffdb53d91d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:06.490238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 546/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:06 2022] GET /identity/v3/role_assignments?user.id=2451d53fb9974222827b6520b8bce5f3&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=3e2c1a2d80234e7fbf13a69c075de868 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:07.577946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259274-14e1-41ac-b73f-7bb96ddacf95 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:07.578527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259274-14e1-41ac-b73f-7bb96ddacf95 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:07.578527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7259274-14e1-41ac-b73f-7bb96ddacf95 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:07.579860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 561/1107] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:07.584881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180079cc-553f-4175-9671-403345612fb2 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:07.585249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180079cc-553f-4175-9671-403345612fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:07.585427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180079cc-553f-4175-9671-403345612fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:07.603785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-180079cc-553f-4175-9671-403345612fb2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:07.604993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-180079cc-553f-4175-9671-403345612fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:07.671299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 547/1108] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:07.936225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7259274-14e1-41ac-b73f-7bb96ddacf95 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:08.033868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7259274-14e1-41ac-b73f-7bb96ddacf95 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:08.068103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579aa1d-c24b-478e-afd3-9c74bddd7382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:08.068721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579aa1d-c24b-478e-afd3-9c74bddd7382 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:08.068998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579aa1d-c24b-478e-afd3-9c74bddd7382 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:08.069269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579aa1d-c24b-478e-afd3-9c74bddd7382 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:08.070388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579aa1d-c24b-478e-afd3-9c74bddd7382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:08.071811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579aa1d-c24b-478e-afd3-9c74bddd7382 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:08.073373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 562/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:08.078998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180079cc-553f-4175-9671-403345612fb2 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:08.129532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:08.175471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180079cc-553f-4175-9671-403345612fb2 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:08.208176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce493dd-82fa-4943-b305-881fd3196d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:08.208780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce493dd-82fa-4943-b305-881fd3196d69 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:08.209059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce493dd-82fa-4943-b305-881fd3196d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:08.209332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce493dd-82fa-4943-b305-881fd3196d69 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:08.209972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce493dd-82fa-4943-b305-881fd3196d69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:08.211420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce493dd-82fa-4943-b305-881fd3196d69 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:08.233003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 548/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:49:08 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:49:10.400367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae9e7-bb03-4b89-b9da-4e36c67cd499 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:10.400897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae9e7-bb03-4b89-b9da-4e36c67cd499 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:10.400897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ae9e7-bb03-4b89-b9da-4e36c67cd499 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:10.402868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 563/1111] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:10.408183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:10.408628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:10.408801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:10.426929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:10.428116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:10.492356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 549/1112] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:10.759395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae9e7-bb03-4b89-b9da-4e36c67cd499 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:10.814544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:10.856696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ae9e7-bb03-4b89-b9da-4e36c67cd499 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:10.889546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:10.890187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:10.890462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:10.890733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] PATH_INFO: `/v3/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:10.892102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] RBAC: Authorizing `identity:get_role(role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:10.893455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:10.894777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 564/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:10 2022] GET /identity/v3/roles/399f7b70c70b4f09addcf156bf604eab => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:10.900485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:10.938323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:10.997875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472a6e7-c8ad-445d-b138-0baa6875ee75 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:11.030589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:11.031212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:11.031490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:11.031761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] PATH_INFO: `/v3/users/ab5ac1dc9c3e430e89d412fda0855241` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:11.045567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=ab5ac1dc9c3e430e89d412fda0855241)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:11.047075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:11.048633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 550/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:10 2022] GET /identity/v3/users/ab5ac1dc9c3e430e89d412fda0855241 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:11.054355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:11.149933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0007997e-b46b-4768-96c0-820be455b509 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:11.172091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:11.182609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:11.183676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:11.183953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:11.184222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:11.250792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:11.252235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:11.345526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 565/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:11.351620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:11.441217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:11.447466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4c756-b341-4fb7-affb-6c9407256bd1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:11.480509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42c997-b035-47d9-a3ba-7d5180eb5c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:11.481118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42c997-b035-47d9-a3ba-7d5180eb5c52 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:11.481388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42c997-b035-47d9-a3ba-7d5180eb5c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:11.481655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42c997-b035-47d9-a3ba-7d5180eb5c52 None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:11.482768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42c997-b035-47d9-a3ba-7d5180eb5c52 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca87a9a417a4d168187240c172b69a0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:11.484177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42c997-b035-47d9-a3ba-7d5180eb5c52 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:11.486229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 551/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:11 2022] GET /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:11.493001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:11.516823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:11.587668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6a502-b333-4f94-8714-e8875623f0c3 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:11.620385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:11.620971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:11.621247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:11.621518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:11.622251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] RBAC: Adding query filter params (role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=6ca87a9a417a4d168187240c172b69a0, user.id=ab5ac1dc9c3e430e89d412fda0855241) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:11.622556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:11.623993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf3455-23f3-4857-93ed-6c48d1aba736 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:11.629164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 566/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:11 2022] GET /identity/v3/role_assignments?user.id=ab5ac1dc9c3e430e89d412fda0855241&scope.project.id=6ca87a9a417a4d168187240c172b69a0&role.id=399f7b70c70b4f09addcf156bf604eab => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:13.732484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da8b78-397c-40d1-827b-1d8efa00574d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:13.732964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da8b78-397c-40d1-827b-1d8efa00574d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:13.733070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da8b78-397c-40d1-827b-1d8efa00574d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:13.734392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 552/1118] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:13.739494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:13.739890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:13.740045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:13.758635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:13.759805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:13.824651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 567/1119] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:14.089262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da8b78-397c-40d1-827b-1d8efa00574d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:14.187081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da8b78-397c-40d1-827b-1d8efa00574d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:14.218896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:14.219502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:14.219782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:14.220051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] PATH_INFO: `/v3/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:14.221401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] RBAC: Authorizing `identity:get_role(role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:14.222775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:14.224029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 553/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:14 2022] GET /identity/v3/roles/399f7b70c70b4f09addcf156bf604eab => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:14.230140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:14.248210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:14.326131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786b4d0-35de-4ed8-a431-4553852716bd None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:14.361095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:14.361652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:14.361929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:14.362231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] PATH_INFO: `/v3/users/ab5ac1dc9c3e430e89d412fda0855241` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:14.363334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] RBAC: Authorizing `identity:get_user(user_id=ab5ac1dc9c3e430e89d412fda0855241)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:14.364743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:14.366333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 568/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:14 2022] GET /identity/v3/users/ab5ac1dc9c3e430e89d412fda0855241 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:14.371929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:14.402349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:14.469025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcee958-5959-4d28-aa9a-913e9a835bbc None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:14.501241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e3292-077e-4f2f-8722-c9b1dbd5cc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:14.501848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e3292-077e-4f2f-8722-c9b1dbd5cc0c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:14.502149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e3292-077e-4f2f-8722-c9b1dbd5cc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:14.502418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e3292-077e-4f2f-8722-c9b1dbd5cc0c None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:14.503503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e3292-077e-4f2f-8722-c9b1dbd5cc0c None admin] RBAC: Authorizing `identity:get_project(project_id=6ca87a9a417a4d168187240c172b69a0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:14.505162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e3292-077e-4f2f-8722-c9b1dbd5cc0c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:14.506766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 554/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:14 2022] GET /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:49:14.512582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:14.593986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:14.609612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247e447-3db9-4ff2-bb16-0e65c00eb5ad None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:14.641541 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd9fa0-4d6d-4c97-8512-aebf689e7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:14.642149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd9fa0-4d6d-4c97-8512-aebf689e7eb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:14.642425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd9fa0-4d6d-4c97-8512-aebf689e7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:14.642694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd9fa0-4d6d-4c97-8512-aebf689e7eb2 None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0/users/ab5ac1dc9c3e430e89d412fda0855241/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:14.644777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd9fa0-4d6d-4c97-8512-aebf689e7eb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ca87a9a417a4d168187240c172b69a0, user_id=ab5ac1dc9c3e430e89d412fda0855241, role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:14.646232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd9fa0-4d6d-4c97-8512-aebf689e7eb2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:14.660399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 569/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:49:14 2022] PUT /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0/users/ab5ac1dc9c3e430e89d412fda0855241/roles/399f7b70c70b4f09addcf156bf604eab => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:49:16.764013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa464bb7-a796-480a-8dba-b80e109e8d6a None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:16.764534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa464bb7-a796-480a-8dba-b80e109e8d6a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:16.764645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa464bb7-a796-480a-8dba-b80e109e8d6a None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:16.765903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 555/1124] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:16.771293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:16.771686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:16.771843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:16.789609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:16.790821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:16.856229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 570/1125] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:17.122478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa464bb7-a796-480a-8dba-b80e109e8d6a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:17.217164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:17.221200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa464bb7-a796-480a-8dba-b80e109e8d6a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:17.252882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:17.253488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:17.253759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:17.254042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] PATH_INFO: `/v3/roles/399f7b70c70b4f09addcf156bf604eab` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:17.255661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] RBAC: Authorizing `identity:get_role(role_id=399f7b70c70b4f09addcf156bf604eab)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:17.257006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:17.258300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 556/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:17 2022] GET /identity/v3/roles/399f7b70c70b4f09addcf156bf604eab => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:17.264569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:17.336694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:17.360675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70a85b-5003-43d7-9106-52c83027c69c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:17.393793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:17.394382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:17.394661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:17.394930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] PATH_INFO: `/v3/users/ab5ac1dc9c3e430e89d412fda0855241` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:17.396029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] RBAC: Authorizing `identity:get_user(user_id=ab5ac1dc9c3e430e89d412fda0855241)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:17.397438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:17.399015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 571/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:17 2022] GET /identity/v3/users/ab5ac1dc9c3e430e89d412fda0855241 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:17.404676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:17.498975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db27d7-72b4-490c-99a7-aea3c0cabd73 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:17.531554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:17.532130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:17.532403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:17.532671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:17.565053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:17.596392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:17.597804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:17.692039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 557/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:17.697976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:17.795032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9de867-7a0f-4ae4-8168-719ada3920b6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:17.827011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9763df-cd73-4be1-ac7e-b4c8f106fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:17.827624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9763df-cd73-4be1-ac7e-b4c8f106fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:17.827901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9763df-cd73-4be1-ac7e-b4c8f106fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:17.828172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9763df-cd73-4be1-ac7e-b4c8f106fbf4 None admin] PATH_INFO: `/v3/projects/6ca87a9a417a4d168187240c172b69a0` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:17.829264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9763df-cd73-4be1-ac7e-b4c8f106fbf4 None admin] RBAC: Authorizing `identity:get_project(project_id=6ca87a9a417a4d168187240c172b69a0)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:17.830712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9763df-cd73-4be1-ac7e-b4c8f106fbf4 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:17.832334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 572/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:17 2022] GET /identity/v3/projects/6ca87a9a417a4d168187240c172b69a0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:17.838495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:17.907544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:17.940150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8f994-1970-46c0-bc88-b2ed8a207730 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:17.971860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:17.972448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:17.972720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:17.972985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:17.973686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] RBAC: Adding query filter params (role.id=399f7b70c70b4f09addcf156bf604eab, scope.project.id=6ca87a9a417a4d168187240c172b69a0, user.id=ab5ac1dc9c3e430e89d412fda0855241) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:17.973983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:17.975412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b8290-0d5c-4a70-8e56-77fbbe8fa2ba None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:17.981403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 558/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:17 2022] GET /identity/v3/role_assignments?user.id=ab5ac1dc9c3e430e89d412fda0855241&scope.project.id=6ca87a9a417a4d168187240c172b69a0&role.id=399f7b70c70b4f09addcf156bf604eab => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:20.121983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ea58e-a10e-4614-882f-5506a4269837 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:20.122569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ea58e-a10e-4614-882f-5506a4269837 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:20.122569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ea58e-a10e-4614-882f-5506a4269837 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:20.123872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 573/1131] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:20.128989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:20.129407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:20.129511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:20.153424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:20.154893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:20.219140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 559/1132] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:49:20.491040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ea58e-a10e-4614-882f-5506a4269837 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:20.508678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:20.592021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ea58e-a10e-4614-882f-5506a4269837 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:20.624510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa4d94-db34-49f7-bfdf-f78bb299077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:20.625121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa4d94-db34-49f7-bfdf-f78bb299077c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:20.625392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa4d94-db34-49f7-bfdf-f78bb299077c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:20.625655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa4d94-db34-49f7-bfdf-f78bb299077c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:20.626760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa4d94-db34-49f7-bfdf-f78bb299077c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:20.628167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa4d94-db34-49f7-bfdf-f78bb299077c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:20.629954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 574/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:20.635759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:20.694320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:20.731662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521053c-dce7-4358-bdaf-04ec724e5f68 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:20.764021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83408d-e321-4539-863a-c361f7005f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:20.764557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83408d-e321-4539-863a-c361f7005f74 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:20.764831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83408d-e321-4539-863a-c361f7005f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:20.765099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83408d-e321-4539-863a-c361f7005f74 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:20.765730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83408d-e321-4539-863a-c361f7005f74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:20.767171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83408d-e321-4539-863a-c361f7005f74 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:20.792825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 560/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:49:20 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:49:22.958298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f58107-8c30-4689-8aad-5fb800b4bd82 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:22.958803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f58107-8c30-4689-8aad-5fb800b4bd82 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:22.958893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f58107-8c30-4689-8aad-5fb800b4bd82 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:22.960186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 575/1135] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:22.965852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:22.966270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:22.966435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:22.984827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:22.986017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:23.054714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 561/1136] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:23.323940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f58107-8c30-4689-8aad-5fb800b4bd82 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:23.404541 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:23.424206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f58107-8c30-4689-8aad-5fb800b4bd82 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:23.457112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c802ea-b7b2-4083-905a-6a19b71b141d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:23.457726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c802ea-b7b2-4083-905a-6a19b71b141d None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:23.458036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c802ea-b7b2-4083-905a-6a19b71b141d None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:23.458348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c802ea-b7b2-4083-905a-6a19b71b141d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:23.459480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c802ea-b7b2-4083-905a-6a19b71b141d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:23.460907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c802ea-b7b2-4083-905a-6a19b71b141d None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:23.462499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 576/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:49:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:23.468630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:23.512871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:23.565581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773ce3a-79b9-4187-86c2-9499194779d1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:23.598595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd934b93-97a7-415c-9307-12720c06e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:23.599137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd934b93-97a7-415c-9307-12720c06e021 None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:23.599414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd934b93-97a7-415c-9307-12720c06e021 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:23.599686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd934b93-97a7-415c-9307-12720c06e021 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:23.600323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd934b93-97a7-415c-9307-12720c06e021 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:23.601746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd934b93-97a7-415c-9307-12720c06e021 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:23.623082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 562/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:49:23 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:49:25.779713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb71d-d42d-4c0b-8b5f-8b66b3516090 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:25.780251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb71d-d42d-4c0b-8b5f-8b66b3516090 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:25.780251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cb71d-d42d-4c0b-8b5f-8b66b3516090 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:25.781565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 577/1139] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:25.786678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:25.787044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:25.787228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:25.806086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:25.807255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:25.870979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 563/1140] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:26.134506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb71d-d42d-4c0b-8b5f-8b66b3516090 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:26.231726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cb71d-d42d-4c0b-8b5f-8b66b3516090 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:26.264344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:26.264960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:26.265236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:26.265507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:26.275259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:26.280595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:26.281706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:26.282781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 578/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:26.287647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:26.316267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:26.386954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdbd2a-814c-4635-9c2d-65b6966e5633 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:26.418362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:26.418950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:26.419222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:26.419488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:26.420136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:26.420429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:26.421778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:26.429422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 564/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:26.435750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:26.446782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:26.532985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8541a7b-9b65-4a89-8136-29fc85cc95fd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:26.565542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:26.566186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:26.566465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:26.566738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] PATH_INFO: `/v3/users/cdb2182e2dd5423095b8a548a7ae2b07` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:26.579917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb2182e2dd5423095b8a548a7ae2b07)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:26.581334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:26.582923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 579/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:26 2022] GET /identity/v3/users/cdb2182e2dd5423095b8a548a7ae2b07 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:26.588557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:26.677711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:26.687108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7551d-787b-44c6-956b-b6d20566bdb3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:26.719692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:26.720286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:26.720563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:26.720833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:26.785303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:26.786759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:26.871421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:26.880945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 565/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:26.887285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:26.949935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:26.983090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a5157-e6a6-41b0-ae2e-50e8c4f49cc6 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:27.015569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822af1f-c879-4a68-9e07-757bc089b6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:27.016184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822af1f-c879-4a68-9e07-757bc089b6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:27.016463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822af1f-c879-4a68-9e07-757bc089b6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:27.016736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822af1f-c879-4a68-9e07-757bc089b6d2 None admin] PATH_INFO: `/v3/projects/44381981a60d4b609548096270eeec94` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:27.017829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822af1f-c879-4a68-9e07-757bc089b6d2 None admin] RBAC: Authorizing `identity:get_project(project_id=44381981a60d4b609548096270eeec94)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:27.019277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822af1f-c879-4a68-9e07-757bc089b6d2 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:27.020881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 580/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:26 2022] GET /identity/v3/projects/44381981a60d4b609548096270eeec94 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:27.027509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:27.121604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35708a70-0abe-432b-8faf-8e1b354701fc None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:27.154033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:27.154614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:27.154888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:27.155157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:27.155861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=44381981a60d4b609548096270eeec94, user.id=cdb2182e2dd5423095b8a548a7ae2b07) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:27.156161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:27.157560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748db13-3b4e-4c68-b41f-7c81de9610c0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:27.162650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 566/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:27 2022] GET /identity/v3/role_assignments?user.id=cdb2182e2dd5423095b8a548a7ae2b07&scope.project.id=44381981a60d4b609548096270eeec94&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:29.248745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78fc69-b93b-4d9c-8a5c-e0feef074de4 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:29.249292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78fc69-b93b-4d9c-8a5c-e0feef074de4 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:29.249292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78fc69-b93b-4d9c-8a5c-e0feef074de4 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:29.250660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 581/1147] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:29.256054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:29.256440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:29.256595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:29.274866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:29.276038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:29.325277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:29.342269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 567/1148] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:29.610263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78fc69-b93b-4d9c-8a5c-e0feef074de4 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:29.708506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78fc69-b93b-4d9c-8a5c-e0feef074de4 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:29.741069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:29.741682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:29.741957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:29.742261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:29.744003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:29.753176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:29.758230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:29.759315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:29.760487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 582/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:29.765304 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:29.862357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a28592-4177-41c0-a61d-b0e1f3dbda81 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:29.895022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:29.895606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:29.895877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:29.896139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:29.896789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:29.897080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:29.898446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:29.906712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 568/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:29.912374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:30.008349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026305-94bd-437f-841c-b01a1d396066 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:30.041066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:30.041673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:30.041948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:30.042263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] PATH_INFO: `/v3/users/cdb2182e2dd5423095b8a548a7ae2b07` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:30.043355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] RBAC: Authorizing `identity:get_user(user_id=cdb2182e2dd5423095b8a548a7ae2b07)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:30.044756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:30.046320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 583/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:29 2022] GET /identity/v3/users/cdb2182e2dd5423095b8a548a7ae2b07 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:30.051966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:30.098849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:30.147782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2ce38-2473-441c-8422-d3c2e1939861 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:30.178714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8570216-b10a-4b25-a5fd-baae1fffe1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:30.179320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8570216-b10a-4b25-a5fd-baae1fffe1ae None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:30.179593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8570216-b10a-4b25-a5fd-baae1fffe1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:30.179860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8570216-b10a-4b25-a5fd-baae1fffe1ae None admin] PATH_INFO: `/v3/projects/44381981a60d4b609548096270eeec94` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:30.180945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8570216-b10a-4b25-a5fd-baae1fffe1ae None admin] RBAC: Authorizing `identity:get_project(project_id=44381981a60d4b609548096270eeec94)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:30.182373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8570216-b10a-4b25-a5fd-baae1fffe1ae None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:30.184453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 569/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:30 2022] GET /identity/v3/projects/44381981a60d4b609548096270eeec94 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:30.190225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:30.252774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:30.291120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f5c7-2ccf-4eaa-bbac-5dd41e964f5c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:30.323589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b6d07-ef91-4c61-8b45-e0d8fa089b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:30.324237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b6d07-ef91-4c61-8b45-e0d8fa089b8c None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:30.324513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b6d07-ef91-4c61-8b45-e0d8fa089b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:30.324786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b6d07-ef91-4c61-8b45-e0d8fa089b8c None admin] PATH_INFO: `/v3/projects/44381981a60d4b609548096270eeec94/users/cdb2182e2dd5423095b8a548a7ae2b07/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:30.326654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b6d07-ef91-4c61-8b45-e0d8fa089b8c None admin] RBAC: Authorizing `identity:create_grant(project_id=44381981a60d4b609548096270eeec94, user_id=cdb2182e2dd5423095b8a548a7ae2b07, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:30.328086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b6d07-ef91-4c61-8b45-e0d8fa089b8c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:30.342723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 584/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:49:30 2022] PUT /identity/v3/projects/44381981a60d4b609548096270eeec94/users/cdb2182e2dd5423095b8a548a7ae2b07/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:49:32.431538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ede697-0624-447c-a342-6d869a8bb113 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:32.432126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ede697-0624-447c-a342-6d869a8bb113 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:32.432126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ede697-0624-447c-a342-6d869a8bb113 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:32.433459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 570/1154] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:32.438723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:32.439142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:32.439273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:32.456941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:32.458150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:32.523202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 585/1155] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:32.792002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ede697-0624-447c-a342-6d869a8bb113 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:32.890963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ede697-0624-447c-a342-6d869a8bb113 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:32.906690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:32.924055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:32.924672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:32.924949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:32.925219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:32.935582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:32.940337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:32.941519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:32.942660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 571/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:32.947356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:32.997592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:33.043241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c54da-b0bf-4c3e-82e3-679d74429247 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:33.075462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:33.076056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:33.076334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:33.076603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:33.077256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:33.077555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:33.078932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:33.086600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 586/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:33.092342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:33.187750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c35cf1-3f20-4f56-a366-39d7140a57d3 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:33.219595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:33.220203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:33.220478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:33.220745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] PATH_INFO: `/v3/users/cdb2182e2dd5423095b8a548a7ae2b07` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:33.221839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=cdb2182e2dd5423095b8a548a7ae2b07)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:33.223272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:33.224800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 572/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:33 2022] GET /identity/v3/users/cdb2182e2dd5423095b8a548a7ae2b07 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:33.230869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:33.328833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbac06-1ad9-434e-ad75-c4e51d7ca324 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:33.342664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:33.363024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:33.363629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:33.363906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:33.364178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:33.431041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:33.432483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:33.525308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 587/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:33.531413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:33.576798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:33.631983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d944a-52a9-43ec-b739-5040aa2e3bf0 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:33.664090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5e43e-472c-469a-8162-648140424096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:33.664719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e43e-472c-469a-8162-648140424096 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:33.664996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e43e-472c-469a-8162-648140424096 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:33.665267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e43e-472c-469a-8162-648140424096 None admin] PATH_INFO: `/v3/projects/44381981a60d4b609548096270eeec94` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:33.666402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5e43e-472c-469a-8162-648140424096 None admin] RBAC: Authorizing `identity:get_project(project_id=44381981a60d4b609548096270eeec94)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:33.668318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5e43e-472c-469a-8162-648140424096 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:33.669924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 573/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:33 2022] GET /identity/v3/projects/44381981a60d4b609548096270eeec94 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:33.676069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:33.691127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:33.771229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7863800-1984-4aa2-aec4-fd6d5197b6ad None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:33.803517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:33.804109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:33.804387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:33.804659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:33.805368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=44381981a60d4b609548096270eeec94, user.id=cdb2182e2dd5423095b8a548a7ae2b07) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:33.805672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:33.807124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f2f0-dcc0-4262-a24a-0d137a3b7cc1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:33.813171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 588/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:33 2022] GET /identity/v3/role_assignments?user.id=cdb2182e2dd5423095b8a548a7ae2b07&scope.project.id=44381981a60d4b609548096270eeec94&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:35.959687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1063-7968-4775-b1e4-68fae371ab6a None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:35.960214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1063-7968-4775-b1e4-68fae371ab6a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:35.960214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1063-7968-4775-b1e4-68fae371ab6a None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:35.961557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 574/1162] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:35.967495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:35.967906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:35.968009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:35.985612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:35.986813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:36.050079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 589/1163] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:36.316726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1063-7968-4775-b1e4-68fae371ab6a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:36.416145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1063-7968-4775-b1e4-68fae371ab6a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:36.428310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:36.449013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:36.449599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:36.449870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:36.450166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:36.515107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:36.516527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:36.610499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 575/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:36.616316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:36.626221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:36.714863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46640-5f6b-48ba-a3c6-e70f7a21cd25 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:36.746628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:36.747239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:36.747513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:36.747779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:36.754121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:36.755778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-b7181e65-9df7-4616-a818-5ced9f36455e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:36.767258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:36.768602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:36.769741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 590/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:49:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:36.774262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:36.788336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:36.868960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1791e9b2-bda6-42a4-b61e-497f6f5ee8e1 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:36.902207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:36.902797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:36.903069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:36.903334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:36.903957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:36.904249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:36.905588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071f331-6a53-499d-b583-7d0c7d5c101c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:36.912390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 576/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Sat Sep 24 04:49:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:38.989754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eb363-53c9-45f2-9386-d3921243400b None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:38.990420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eb363-53c9-45f2-9386-d3921243400b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:38.990420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239eb363-53c9-45f2-9386-d3921243400b None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:38.991671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 591/1167] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:38.997433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779500d1-b1fd-4789-9beb-335ded749d6c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:38.997858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779500d1-b1fd-4789-9beb-335ded749d6c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:38.997956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779500d1-b1fd-4789-9beb-335ded749d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:39.015834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-779500d1-b1fd-4789-9beb-335ded749d6c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:39.016992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-779500d1-b1fd-4789-9beb-335ded749d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:39.080520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 577/1168] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:39.345819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eb363-53c9-45f2-9386-d3921243400b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:39.410521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:39.443540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239eb363-53c9-45f2-9386-d3921243400b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:39.478007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0c50a-1eaa-4cd5-a4c1-5c7cb7278758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:39.478621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0c50a-1eaa-4cd5-a4c1-5c7cb7278758 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:39.478898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0c50a-1eaa-4cd5-a4c1-5c7cb7278758 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:39.479166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0c50a-1eaa-4cd5-a4c1-5c7cb7278758 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:39.479833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0c50a-1eaa-4cd5-a4c1-5c7cb7278758 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:39.481811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc0c50a-1eaa-4cd5-a4c1-5c7cb7278758 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:39.496944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 592/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Sat Sep 24 04:49:39 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:49:41.655698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0ea83-51c7-4a7f-86ad-4b36fde3e63e None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:41.656140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0ea83-51c7-4a7f-86ad-4b36fde3e63e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:41.656233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0ea83-51c7-4a7f-86ad-4b36fde3e63e None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:41.657572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 578/1170] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:41.662931 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:41.663331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:41.663482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:41.681772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:41.682984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:41.746513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 593/1171] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:42.016622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0ea83-51c7-4a7f-86ad-4b36fde3e63e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:42.025427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:42.115318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0ea83-51c7-4a7f-86ad-4b36fde3e63e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:42.148727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbee962-2484-4de8-b3ca-1d3528b9420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:42.149340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee962-2484-4de8-b3ca-1d3528b9420d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:42.149616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee962-2484-4de8-b3ca-1d3528b9420d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:42.149885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbee962-2484-4de8-b3ca-1d3528b9420d None admin] PATH_INFO: `/v3/domains/0fcc78380de34350bac54202375ee5a8` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:42.159955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbee962-2484-4de8-b3ca-1d3528b9420d None admin] RBAC: Authorizing `identity:get_domain(domain_id=0fcc78380de34350bac54202375ee5a8)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:42.161377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbee962-2484-4de8-b3ca-1d3528b9420d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:42.162939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 579/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Sat Sep 24 04:49:42 2022] GET /identity/v3/domains/0fcc78380de34350bac54202375ee5a8 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:42.168663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:42.225172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:42.266423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52478-009a-4ad8-88c6-a9cbdc3aac69 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:42.298182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5afcfd-d919-4050-a3cb-b4ef55410f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:42.298725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5afcfd-d919-4050-a3cb-b4ef55410f91 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:42.299001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5afcfd-d919-4050-a3cb-b4ef55410f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:42.299267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5afcfd-d919-4050-a3cb-b4ef55410f91 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:42.299907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5afcfd-d919-4050-a3cb-b4ef55410f91 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:42.301319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5afcfd-d919-4050-a3cb-b4ef55410f91 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:42.327107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 594/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:49:42 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:49:44.475104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8dfd4-9d5d-4165-8cc6-60166b8cfaf8 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:44.475631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8dfd4-9d5d-4165-8cc6-60166b8cfaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:44.475631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8dfd4-9d5d-4165-8cc6-60166b8cfaf8 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:44.476964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 580/1174] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:44.482773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:44.483166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:44.483324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:44.501953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:44.503248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:44.567936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 595/1175] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:44.831467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8dfd4-9d5d-4165-8cc6-60166b8cfaf8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:44.895750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:44.930587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8dfd4-9d5d-4165-8cc6-60166b8cfaf8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:44.962278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53389c-f993-4298-954f-e54bae3bde3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:44.962889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53389c-f993-4298-954f-e54bae3bde3d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:44.963159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53389c-f993-4298-954f-e54bae3bde3d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:44.963427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53389c-f993-4298-954f-e54bae3bde3d None admin] PATH_INFO: `/v3/domains/0fcc78380de34350bac54202375ee5a8` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:44.964527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53389c-f993-4298-954f-e54bae3bde3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=0fcc78380de34350bac54202375ee5a8)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:44.965934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53389c-f993-4298-954f-e54bae3bde3d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:44.967476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 581/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Sat Sep 24 04:49:44 2022] GET /identity/v3/domains/0fcc78380de34350bac54202375ee5a8 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:44.973154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:45.013847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:45.069891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38534332-ef46-4c1a-be53-84beb6688c22 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:45.103138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aed1a-6fb2-4ff6-a6fc-330f38d1e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:45.103681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aed1a-6fb2-4ff6-a6fc-330f38d1e567 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:45.103959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aed1a-6fb2-4ff6-a6fc-330f38d1e567 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:45.104230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aed1a-6fb2-4ff6-a6fc-330f38d1e567 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:45.104881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aed1a-6fb2-4ff6-a6fc-330f38d1e567 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:45.106360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aed1a-6fb2-4ff6-a6fc-330f38d1e567 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:45.128210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 596/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:49:44 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:49:47.270900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255cf0-1e9e-411a-b147-3de044914083 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:47.271459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255cf0-1e9e-411a-b147-3de044914083 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:47.271459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255cf0-1e9e-411a-b147-3de044914083 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:47.272745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 582/1178] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:47.277859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:47.278305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:47.278440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:47.296571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:47.297749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:47.362613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 597/1179] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:47.628015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255cf0-1e9e-411a-b147-3de044914083 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:47.726985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255cf0-1e9e-411a-b147-3de044914083 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:47.759078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:47.759686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:47.759957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:47.760219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:47.762532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:47.771666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:47.776367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:47.778110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:47.779380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 583/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:47.784006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:47.809471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:47.879644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30574c14-fb7d-4128-8008-c1ce0364be7c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:47.913292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:47.913892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:47.914256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:47.914536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:47.915193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:47.915492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:47.916846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:47.924572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 598/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:47.930275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:48.029706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9abb2-421e-43eb-8d41-060af794c496 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:48.061164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:48.061799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:48.062102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:48.062372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] PATH_INFO: `/v3/users/beacb0e07ee541b5a60f9d4c9e935dbf` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:48.075312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] RBAC: Authorizing `identity:get_user(user_id=beacb0e07ee541b5a60f9d4c9e935dbf)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:48.076732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:48.078308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 584/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:47 2022] GET /identity/v3/users/beacb0e07ee541b5a60f9d4c9e935dbf => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:48.084326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:48.167751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:48.179492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc70f76-d453-4b3f-93c3-af9fa9008c13 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:48.211263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:48.211846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:48.212122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:48.212389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:48.276897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:48.278357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:48.358362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:48.370471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 599/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:48.376959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:48.427609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:48.472881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb587fa-7a7f-4fcf-87de-9bd2e4034328 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:48.505868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eda16e-e33b-4d48-a76f-ec7033c6eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:48.506763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eda16e-e33b-4d48-a76f-ec7033c6eea2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:48.507040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eda16e-e33b-4d48-a76f-ec7033c6eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:48.507307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eda16e-e33b-4d48-a76f-ec7033c6eea2 None admin] PATH_INFO: `/v3/projects/0208910d48dc4364b5cfd87a61212be5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:48.508406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eda16e-e33b-4d48-a76f-ec7033c6eea2 None admin] RBAC: Authorizing `identity:get_project(project_id=0208910d48dc4364b5cfd87a61212be5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:48.509818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eda16e-e33b-4d48-a76f-ec7033c6eea2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:48.511801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 585/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:48 2022] GET /identity/v3/projects/0208910d48dc4364b5cfd87a61212be5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:48.517819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:48.618642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931ba73-ae52-4115-939d-8fde10e88458 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:48.653692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:48.654321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:48.654602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:48.654875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:48.655590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=0208910d48dc4364b5cfd87a61212be5, user.id=beacb0e07ee541b5a60f9d4c9e935dbf) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:48.655898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:48.657328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67077e-b556-4320-a6d1-f1ffc5ffd4ce None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:48.662689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 600/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:48 2022] GET /identity/v3/role_assignments?user.id=beacb0e07ee541b5a60f9d4c9e935dbf&scope.project.id=0208910d48dc4364b5cfd87a61212be5&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:50.769602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c75b-b4eb-453b-922a-68782489e953 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:50.770173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c75b-b4eb-453b-922a-68782489e953 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:50.770173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83c75b-b4eb-453b-922a-68782489e953 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:50.771484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 586/1186] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:50.776702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db874-ee85-442a-91d7-71890444d348 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:50.777645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db874-ee85-442a-91d7-71890444d348 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:50.777838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db874-ee85-442a-91d7-71890444d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:50.795471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-760db874-ee85-442a-91d7-71890444d348 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:50.796640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-760db874-ee85-442a-91d7-71890444d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:50.842594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:50.862133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 601/1187] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:51.126259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83c75b-b4eb-453b-922a-68782489e953 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:51.225433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83c75b-b4eb-453b-922a-68782489e953 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:51.244763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:51.259251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:51.259867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:51.260139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:51.260404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:51.270902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:51.275619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:51.276682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:49:51.278908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 587/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:51.282776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760db874-ee85-442a-91d7-71890444d348 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:51.376085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760db874-ee85-442a-91d7-71890444d348 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:51.408598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:51.409137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:51.409415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:51.409687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:51.410368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:51.410666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:51.412014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:51.419664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 602/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:51.425733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:51.521545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140a18b-5c9d-4802-8747-16ea17b77454 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:51.553468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:51.554093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:51.554366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:51.554630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] PATH_INFO: `/v3/users/beacb0e07ee541b5a60f9d4c9e935dbf` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:51.555712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] RBAC: Authorizing `identity:get_user(user_id=beacb0e07ee541b5a60f9d4c9e935dbf)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:51.557110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:51.558747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 588/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:51 2022] GET /identity/v3/users/beacb0e07ee541b5a60f9d4c9e935dbf => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:51.564231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:51.609574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:51.663850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ff972-a581-4260-9fe7-80647f41398a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:51.695659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452235be-2951-4e4e-8dc4-bdbcfeff6da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:51.696282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452235be-2951-4e4e-8dc4-bdbcfeff6da8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:51.696561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452235be-2951-4e4e-8dc4-bdbcfeff6da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:51.696831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452235be-2951-4e4e-8dc4-bdbcfeff6da8 None admin] PATH_INFO: `/v3/projects/0208910d48dc4364b5cfd87a61212be5` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:51.697933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452235be-2951-4e4e-8dc4-bdbcfeff6da8 None admin] RBAC: Authorizing `identity:get_project(project_id=0208910d48dc4364b5cfd87a61212be5)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:51.700014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452235be-2951-4e4e-8dc4-bdbcfeff6da8 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:51.701625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 603/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:51 2022] GET /identity/v3/projects/0208910d48dc4364b5cfd87a61212be5 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:51.707515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:51.754882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:51.804266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0311a2-0eff-4f7a-9d35-63b7587c970e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:51.836894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2321ed-29d6-4ef9-bec5-db7aa6d8b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:51.837535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2321ed-29d6-4ef9-bec5-db7aa6d8b914 None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:51.837815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2321ed-29d6-4ef9-bec5-db7aa6d8b914 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:51.838119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2321ed-29d6-4ef9-bec5-db7aa6d8b914 None admin] PATH_INFO: `/v3/projects/0208910d48dc4364b5cfd87a61212be5/users/beacb0e07ee541b5a60f9d4c9e935dbf/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:51.840848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2321ed-29d6-4ef9-bec5-db7aa6d8b914 None admin] RBAC: Authorizing `identity:create_grant(project_id=0208910d48dc4364b5cfd87a61212be5, user_id=beacb0e07ee541b5a60f9d4c9e935dbf, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:51.842292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2321ed-29d6-4ef9-bec5-db7aa6d8b914 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:51.854286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 589/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:49:51 2022] PUT /identity/v3/projects/0208910d48dc4364b5cfd87a61212be5/users/beacb0e07ee541b5a60f9d4c9e935dbf/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:49:53.992046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b653a-2086-4d02-93e2-d06cbb094609 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:53.992575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b653a-2086-4d02-93e2-d06cbb094609 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:53.992575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b653a-2086-4d02-93e2-d06cbb094609 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:53.993920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 604/1193] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:49:53.999639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:54.000054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:54.000162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:54.017935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:49:54.019122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:49:54.083544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 590/1194] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:49:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:49:54.351265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b653a-2086-4d02-93e2-d06cbb094609 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:54.449080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b653a-2086-4d02-93e2-d06cbb094609 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:54.461446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:54.482336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:54.482948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:54.483226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:54.483496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:54.493217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:49:54.497986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:49:54.499179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:49:54.500024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 605/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:49:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:49:54.504943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:54.547067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:54.606647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc568c-4917-4ea6-bad3-c8dfbcfa43c3 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:54.639232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:54.639825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:54.640098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:54.640362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:54.641012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:54.641305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:54.642732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:54.651124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 591/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:49:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:54.657585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:54.752343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50441e6-1496-4764-87d0-e5a9fd615efd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:54.785540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:54.786187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:54.786462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:54.786728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] PATH_INFO: `/v3/users/beacb0e07ee541b5a60f9d4c9e935dbf` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:54.787819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] RBAC: Authorizing `identity:get_user(user_id=beacb0e07ee541b5a60f9d4c9e935dbf)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:54.789236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:54.790822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 606/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Sat Sep 24 04:49:54 2022] GET /identity/v3/users/beacb0e07ee541b5a60f9d4c9e935dbf => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:54.796540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:54.891052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:54.894029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fd510-a4ce-4260-96ca-36c88644f194 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:54.926677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:54.927266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:54.927541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:54.927810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:54.991468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:54.992881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:55.083645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:55.087109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 592/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:49:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:49:55.093546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:55.134277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:49:55.191077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d059e29-3e64-45ed-a2fb-0905e48f1657 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:55.223392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34934efe-96b6-447d-ba27-39ae432b386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:55.224005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34934efe-96b6-447d-ba27-39ae432b386a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:55.224287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34934efe-96b6-447d-ba27-39ae432b386a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:55.224563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34934efe-96b6-447d-ba27-39ae432b386a None admin] PATH_INFO: `/v3/projects/0208910d48dc4364b5cfd87a61212be5` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:55.225669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34934efe-96b6-447d-ba27-39ae432b386a None admin] RBAC: Authorizing `identity:get_project(project_id=0208910d48dc4364b5cfd87a61212be5)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:55.227120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34934efe-96b6-447d-ba27-39ae432b386a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:55.228724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 607/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Sat Sep 24 04:49:55 2022] GET /identity/v3/projects/0208910d48dc4364b5cfd87a61212be5 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:49:55.235223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:49:55.329534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95721ea5-93ce-489d-a15e-0e6f8c383969 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:49:55.362166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:49:55.362758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:49:55.363032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:49:55.363306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:49:55.364016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=0208910d48dc4364b5cfd87a61212be5, user.id=beacb0e07ee541b5a60f9d4c9e935dbf) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:49:55.364319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:49:55.365733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7af171-c386-4afc-a50d-51fd2afb41a0 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:49:55.371250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 593/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:49:55 2022] GET /identity/v3/role_assignments?user.id=beacb0e07ee541b5a60f9d4c9e935dbf&scope.project.id=0208910d48dc4364b5cfd87a61212be5&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:50:05.219323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=46 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:05.368697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=68 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:51.010460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df6c88-d7be-4868-b5bf-f33977ff9022 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:51.011306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df6c88-d7be-4868-b5bf-f33977ff9022 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:51.011306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df6c88-d7be-4868-b5bf-f33977ff9022 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:51.012356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 608/1201] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:50:51.019593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:51.019593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:51.019593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:51.038434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:50:51.039631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:50:51.122925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 594/1202] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:50:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:50:51.414701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df6c88-d7be-4868-b5bf-f33977ff9022 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:51.543116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df6c88-d7be-4868-b5bf-f33977ff9022 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:51.585626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:51.586294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:51.586575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:51.586887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:51.597918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:51.599531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:50:51.615905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:50:51.617230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:50:51.618254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:50:51.618254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:50:51.618254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:50:51.618254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 609/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:50:51.624259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:51.769108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d3c05-114b-4d75-98a4-cdc780d499a9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:51.802151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:51.809020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:51.809740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:51.810031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:51.810326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:51.811059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:51.811285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:51.812721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9a279-56b0-44ec-bf69-669de6fbd192 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:51.828888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 595/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:50:51.834224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:51.893382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:51.949106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c61be4-805d-45b5-87d6-adfdfe60d32a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:51.986711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f4ac7-de5f-4611-a19e-babcd0153e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:51.987331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4ac7-de5f-4611-a19e-babcd0153e5c None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:51.987609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4ac7-de5f-4611-a19e-babcd0153e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:51.987881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f4ac7-de5f-4611-a19e-babcd0153e5c None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:51.988551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4ac7-de5f-4611-a19e-babcd0153e5c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:51.990024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f4ac7-de5f-4611-a19e-babcd0153e5c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:52.017833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 610/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Sat Sep 24 04:50:51 2022] POST /identity/v3/users => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:50:54.546981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8dfc7-9e28-47a9-ade0-c3375ae66e5d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:54.547552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8dfc7-9e28-47a9-ade0-c3375ae66e5d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:54.547552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8dfc7-9e28-47a9-ade0-c3375ae66e5d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:54.549057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 596/1206] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:50:54.555610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:54.556023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:54.556439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:54.584388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:50:54.586159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:50:54.678743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 611/1207] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:50:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:50:54.978434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8dfc7-9e28-47a9-ade0-c3375ae66e5d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:55.098590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8dfc7-9e28-47a9-ade0-c3375ae66e5d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:55.139809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:55.140787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:55.141044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:55.141302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:55.154748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:50:55.159665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:50:55.164958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:50:55.166263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 597/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:50:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:50:55.167177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:55.211875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:55.282434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf55481-54a0-4e85-8d5c-7c2343d36182 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:55.320905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:55.322206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:55.322206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:55.322206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:55.322717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:55.323016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:55.324383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:55.333565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 612/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:50:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:50:55.339243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:55.403556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:55.478267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195b232-0b9f-4e63-8252-b0c026b29bdd None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:55.514989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:55.515669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:55.515974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:55.516253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:55.529535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:55.531224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:50:55.547316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:50:55.548821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:50:55.549720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:50:55.549720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:50:55.549720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:50:55.549720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 598/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:50:55.554153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:55.660279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:55.664508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39bd33-f797-4742-862f-237ab411f600 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:55.701910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:55.702555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:55.702828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:55.703104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:55.703711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:55.704010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:55.705364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:55.717356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 613/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:50:55.722943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:55.827413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905ab0e-ef1e-47ba-a018-7a4540902875 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:55.828966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:55.866780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:55.867453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:55.867732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:55.867995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:55.876352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:55.877815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:50:55.890143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:50:55.891160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:50:55.892165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:50:55.897008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:55.897522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 599/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:50:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:50:56.000750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9beb474-5a5a-4eda-94c6-2c6624aa4ee5 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:56.037123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:56.037723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:56.038013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:56.038284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:56.039003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:56.039301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:56.040711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:56.059729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 614/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Sat Sep 24 04:50:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:50:56.065465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:56.179523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63374d8-f487-4eee-bb94-1bc029b9fec7 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:56.214736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:56.215438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:56.215713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:56.216585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:56.248494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:56.289022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:56.290695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:56.425314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 600/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:50:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 24 04:50:56.427331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:56.448791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:56.549393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85978758-0080-46bd-a76d-b3c59a312895 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:56.593536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:56.594846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:56.594846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:56.594846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:56.609656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:56.611184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:50:56.630341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:50:56.631511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:50:56.632605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:50:56.632605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:50:56.632605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:50:56.636408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 615/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:50:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:50:56.638259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:56.711251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:56.757860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacdae2-a664-4c88-97aa-3c2d48e3184c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:56.796020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:56.796634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:56.796915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:56.797188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:56.797870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:56.798302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:56.799822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:56.812924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 601/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:50:56.818409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:56.894512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:56.929712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526695e-c714-4144-86a7-43e92f598734 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:56.964519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:56.965056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:56.965331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:56.965600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:56.972979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:56.974465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:50:56.986858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:50:56.987850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:50:56.988758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 616/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:50:56.994075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:57.103156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61ae5b-c357-43fc-8460-4a48ffe11338 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:57.127548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:57.141177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:57.141828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:57.142130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:57.142510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:57.143337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:57.143749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:57.145416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7138370d-338b-49c5-b0e0-6a104b82191d None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:57.160175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 602/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:50:57.166663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:50:57.271762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a8a2d-f224-4d90-9742-02d01fd306d1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:50:57.300941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:50:57.307616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:50:57.308741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:57.309030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:57.309336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:57.310237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=fdf673c05e4443e8bce9b9d7371d1fd7) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:50:57.310606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:50:57.312114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494818d7-cff9-4879-8e21-c64268554058 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:50:57.318213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 617/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:50:57 2022] GET /identity/v3/role_assignments?user.id=fdf673c05e4443e8bce9b9d7371d1fd7&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:50:59.604904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f349120-2d2e-4bb4-8b3e-35194a944c8d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:59.605532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f349120-2d2e-4bb4-8b3e-35194a944c8d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:59.605532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f349120-2d2e-4bb4-8b3e-35194a944c8d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:59.607519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 603/1220] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:50:59.613038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:50:59.613472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:50:59.613612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:50:59.632813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:50:59.634263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:50:59.714930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 618/1221] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:50:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:51:00.014329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f349120-2d2e-4bb4-8b3e-35194a944c8d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:00.123216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f349120-2d2e-4bb4-8b3e-35194a944c8d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:00.159736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:00.160332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:00.160634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:00.161022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:00.172792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:51:00.178235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:00.179386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:00.180383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 604/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:51:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:51:00.188843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:00.304906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb1014-2ceb-488c-8e09-402cdae8ea5c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:00.330136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:00.343688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:00.344383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:00.344660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:00.344930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:00.345757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:00.346099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:00.347484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:00.356503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 619/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:51:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:51:00.363682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:00.418976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:00.482360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6dfa0-8639-442c-8c4d-c4ea15820f51 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:00.520626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:00.521255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:00.521530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:00.521796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:00.532210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:00.534206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:00.551257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:00.557297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:00.558586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:00.558586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:00.558586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:00.558586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 605/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:00.558862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:00.663961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e51ad87-b1ad-4de6-8a92-5a61033cab80 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:00.705500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:00.706138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:00.706416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:00.706679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:00.707350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:00.707639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:00.708950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:00.720881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 620/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:00.727198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:00.832164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:00.839579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f08ade-85a6-46dd-bdcd-9f3edbf2d41a None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:00.877669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:00.878488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:00.878984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:00.879463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:00.888743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:00.890480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:51:00.903327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:00.905013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:00.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 606/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:00.911720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:00.965133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:01.022947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd0645-dbcd-4dfc-80ff-590c3408e10c None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:01.058387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:01.059031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:01.059271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:01.059554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:01.060817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:01.061123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:01.062674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:01.081459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 621/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Sat Sep 24 04:51:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:01.087967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:01.190045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5629e-dbd4-45e6-a35f-8df5db71866c None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:01.225154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:01.225795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:01.226062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:01.226331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:01.237681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:01.239209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:01.253387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:01.254593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:01.255571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:01.255571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:01.255571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:01.255571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 607/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:01.261533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:01.386086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e029c9-9857-4a99-89d2-1e7ff128b599 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:01.389344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:01.429581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:01.430589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:01.430942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:01.431294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:01.432210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:01.432652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:01.434754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:01.452707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 622/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:01.458887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:01.469122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:01.561921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e3c76-2ae8-4736-8bd4-bb96ec6d5090 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:01.596766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:01.597386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:01.597659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:01.597931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:01.604378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:01.605808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:51:01.616882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:01.617970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:01.618912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 608/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:01.624584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:01.732490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c921ee-412c-4b48-96e8-5059e512f259 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:01.773658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:01.774327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:01.774623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:01.774952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:01.775731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:01.776032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:01.777659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc27465-28f6-4f5b-a044-bf5e7138992c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:01.791867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 623/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:01.798106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:01.870312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:01.907877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6c0cd-80b1-4a99-a69e-eab05c505ab8 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:01.943011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f419ec-04d2-425d-a0e3-1a5fa974cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:01.943813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f419ec-04d2-425d-a0e3-1a5fa974cdca None admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:01.944184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f419ec-04d2-425d-a0e3-1a5fa974cdca None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:01.944455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f419ec-04d2-425d-a0e3-1a5fa974cdca None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/fdf673c05e4443e8bce9b9d7371d1fd7/roles/e0ecb2749bfa4e438a98f3f5d4388e80` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:01.961195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f419ec-04d2-425d-a0e3-1a5fa974cdca None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=fdf673c05e4443e8bce9b9d7371d1fd7, role_id=e0ecb2749bfa4e438a98f3f5d4388e80)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:01.962701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f419ec-04d2-425d-a0e3-1a5fa974cdca None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:01.975321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 609/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:51:01 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/fdf673c05e4443e8bce9b9d7371d1fd7/roles/e0ecb2749bfa4e438a98f3f5d4388e80 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:51:04.315480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c227-8f37-45d1-a927-a74a1c32bb6e None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:04.316128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c227-8f37-45d1-a927-a74a1c32bb6e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:04.316128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7c227-8f37-45d1-a927-a74a1c32bb6e None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:04.317854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 624/1233] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:04.323642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:04.324205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:04.324332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:04.343386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:04.344589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:04.434393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 610/1234] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 24 04:51:04.765262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7c227-8f37-45d1-a927-a74a1c32bb6e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:04.796538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:04.883371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7c227-8f37-45d1-a927-a74a1c32bb6e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:04.921267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:04.921803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:04.922136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:04.922384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:04.935142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:51:04.941859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:04.942965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 24 04:51:04.944015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:04.946062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 625/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Sat Sep 24 04:51:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:04.951118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:05.003936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:05.066680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1f734-4252-4023-954f-1db2c9bff1b7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:05.106440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:05.107638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:05.108014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:05.108373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:05.109044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] RBAC: Adding query filter params (name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:05.109370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:05.110755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:05.119942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 611/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Sat Sep 24 04:51:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:05.126639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:05.221148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:05.238957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeae5f6-df38-465d-ac32-ccd204c1f39e None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:05.275854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:05.276477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:05.276751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:05.277024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:05.287073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:05.288529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:05.303575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:05.304736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:05.305840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:05.305840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:05.305840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:05.305840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 626/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:05.311162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:05.422764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796f7b6c-b0c5-4167-b3a2-09af7d3b3453 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:05.431676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:05.457448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:05.458075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:05.458372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:05.458617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:05.459247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:05.459538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:05.460881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:05.473465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 612/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:05.479132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:05.584527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54100e61-fc9c-4245-a6c3-7308b66f0fe8 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:05.622911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:05.623586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:05.623869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:05.624140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:05.627343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:05.633766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:05.635861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:51:05.648457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:05.649571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:05.651151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 627/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:05.656130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:05.765303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ded2a5-12a7-4648-84b9-85de1fa1e483 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:05.805973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:05.806605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:05.806879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:05.807198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:05.807889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:05.808208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:05.809601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:05.830144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 613/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Sat Sep 24 04:51:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:05.836617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:05.965771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f817-019d-4a3a-9004-f28d9c18c437 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:06.000785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:06.001388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:06.001664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:06.001988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:06.069401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:06.077193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:06.078870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:06.191362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 628/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:51:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:51:06.198039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:06.229651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:06.310544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35655dde-45b9-468f-8a57-e22632409593 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:06.349147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:06.349691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:06.350054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:06.350342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:06.361390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:06.363186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:06.381544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:06.387683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:06.389414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:06.389414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:06.389414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:06.389414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 614/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 24 04:51:06.389713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:06.490689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:06.499639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087c313-5031-44d5-bae8-7e31963e8e44 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:06.538593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:06.539207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:06.539483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:06.539774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:06.540482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:06.540881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:06.542457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:06.554768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 629/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:06.561874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:06.652528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:06.676193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb86f6-83e0-4649-8332-7f65c1501e1e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:06.714001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:06.714657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:06.715023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:06.715381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:06.723102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:06.726627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:51:06.741286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:06.742305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:06.743207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 615/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:51:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:06.748821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:06.867275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43480db-74b6-47e5-8290-ad7a4372fd1a None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:06.907567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:06.908108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:06.908396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:06.908727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:06.909532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:06.909837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:06.911548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801857cb-0b83-4d17-97a1-f82441d851bb None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:06.924897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 630/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:51:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:06.931013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:07.045836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d000aa-2a3d-4c2d-943b-bbc0d10f3efd None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:07.087072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:07.087704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:07.087985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:07.088297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:07.089193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] RBAC: Adding query filter params (role.id=e0ecb2749bfa4e438a98f3f5d4388e80, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=fdf673c05e4443e8bce9b9d7371d1fd7) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:07.089542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:07.091223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1556daa-78f2-4012-b44e-2cf5f48e880e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:07.098289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 616/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:51:06 2022] GET /identity/v3/role_assignments?user.id=fdf673c05e4443e8bce9b9d7371d1fd7&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=e0ecb2749bfa4e438a98f3f5d4388e80 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:09.615100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e85a1-bfde-4f96-a11c-1aaa1ffe0d91 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:09.615635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e85a1-bfde-4f96-a11c-1aaa1ffe0d91 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:09.615635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e85a1-bfde-4f96-a11c-1aaa1ffe0d91 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:09.616973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 631/1247] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:09.623191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:09.623633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:09.623902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:09.630892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:09.646468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:09.647904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:09.727375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 617/1248] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:09.998579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e85a1-bfde-4f96-a11c-1aaa1ffe0d91 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:10.017980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:10.107491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e85a1-bfde-4f96-a11c-1aaa1ffe0d91 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:10.142549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:10.143198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:10.143476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:10.143748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:10.155178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:51:10.160105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:10.161255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:10.162617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 632/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:10.167490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:10.269882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed816a1-96a2-43b9-bf94-43cd21badb79 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:10.300169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:10.306020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:10.306642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:10.306914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:10.307178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:10.307842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:10.308144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:10.309494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:10.318492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 618/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:51:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:10.324489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:10.415619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:10.436217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54f46f-9d48-4b69-899a-de9501dbc328 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:10.471108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:10.471741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:10.472016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:10.472287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:10.482221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:10.483649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:10.497913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:10.499139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:10.500421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:10.500421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:10.500421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:10.500421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 633/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:10.506430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:10.606654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e93861-8221-41db-9655-e742228f9d1b None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:10.639228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:10.639798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:10.640076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:10.640342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:10.640971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:10.641263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:10.642635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:10.655909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 619/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:10.661478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:10.756125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407b46d-162f-44f9-b521-bff44693e651 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:10.781061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:10.789038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:10.789652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:10.789924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:10.790220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:10.797253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:10.798704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:51:10.809474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:10.811038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:10.812139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 634/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:10.816807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:10.871951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:10.918858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6d8ed-f56f-4172-b438-04d076bcaed2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:10.952012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:10.952607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:10.952884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:10.953155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:10.953874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:10.954206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:10.955629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:10.972014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 620/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Sat Sep 24 04:51:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:10.977873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:11.074058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb5a06-c20f-4506-8dc9-a97272550787 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:11.105611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:11.106221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:11.106498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:11.107318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:11.158940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:11.172521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:11.173955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:11.267249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 635/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:51:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:51:11.273103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:11.375680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f00509-155e-467b-91a7-5a6de40e367f None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:11.389524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:11.410455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f313-b656-4002-917d-1925759178be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:11.411010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f313-b656-4002-917d-1925759178be None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:11.411281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f313-b656-4002-917d-1925759178be None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:11.411557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2f313-b656-4002-917d-1925759178be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:11.421722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2f313-b656-4002-917d-1925759178be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:11.423177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2f313-b656-4002-917d-1925759178be None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-c7b2f313-b656-4002-917d-1925759178be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:11.437264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:11.438457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:11.439822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:11.439822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:11.439822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:11.439822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 621/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:11.444054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:11.526191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:11.538435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06d582-5c8b-4f20-bd06-f26ab809edc7 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:11.570921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:11.571515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:11.571795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:11.572066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:11.572696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:11.572993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:11.574373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:11.585146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 636/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:11.590635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f313-b656-4002-917d-1925759178be None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:11.688461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2f313-b656-4002-917d-1925759178be None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:11.720322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:11.720933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:11.721213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:11.721480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:11.727892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:11.729302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:51:11.740756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:11.741782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:11.743069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 622/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:11.747624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:11.842866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142deb8-43b7-4022-8649-ab07f69e07dd None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:11.873791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:11.874432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:11.874704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:11.874971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:11.875642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:11.875937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:11.877338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749768-bec6-4c04-a049-2647a7505396 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:11.889610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 637/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:11.895953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:11.922264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:11.997346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c57e30-e6ed-4f8e-9866-8b80f3c97242 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:12.031521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:12.032137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:12.032420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:12.032717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:12.033500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=fdf673c05e4443e8bce9b9d7371d1fd7) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:12.033830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:12.035421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4e927-a95e-4141-8b91-2780586d2872 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:12.040902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 623/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:51:11 2022] GET /identity/v3/role_assignments?user.id=fdf673c05e4443e8bce9b9d7371d1fd7&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:14.142587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700bbcf-8426-4c3c-b62e-f73052d476b5 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:14.143107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700bbcf-8426-4c3c-b62e-f73052d476b5 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:14.143107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700bbcf-8426-4c3c-b62e-f73052d476b5 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:14.144454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 638/1261] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:14.149553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:14.150022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:14.150117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:14.167885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:14.169053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:14.233790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 624/1262] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:14.522242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700bbcf-8426-4c3c-b62e-f73052d476b5 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:14.533423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:14.622532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700bbcf-8426-4c3c-b62e-f73052d476b5 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:14.656626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:14.657255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:14.657535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:14.657807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:14.667878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:51:14.673265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:14.674386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:14.675360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 639/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:51:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:14.682201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:14.700076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:14.779563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f41b4a-3750-447f-a6ee-404ee45be1fd None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:14.811780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:14.812379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:14.812655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:14.812943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:14.813595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:14.813891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:14.815275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:14.823026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 625/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:51:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:14.829164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:14.893855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:14.926188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333aada-35fb-4518-9517-af03cb453002 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:14.958582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:14.959197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:14.959474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:14.959742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:14.969859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:14.971339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:14.985305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:14.986448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:14.987585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:14.987585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:14.987585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:14.987585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 640/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:14.992223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:15.071557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:15.091248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb54eff-520c-49ee-9a5e-e5546c8d0b0e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:15.123951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:15.124542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:15.124816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:15.125098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:15.125706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:15.126027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:15.127359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:15.139310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 626/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:15.144914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:15.239716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19c267-d43a-4612-95f4-01e986ad5962 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:15.258480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:15.272715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:15.273326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:15.273604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:15.273876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:15.281460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:15.282915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:51:15.293744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:15.295440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:15.296731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 641/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:15.300953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:15.404944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50218-e8e5-46b5-bcf7-aa251c8dbcc5 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:15.439858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:15.440453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:15.440752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:15.441056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:15.441820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:15.442154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:15.443571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:15.452181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:15.460540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 627/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Sat Sep 24 04:51:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:15.467114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:15.568118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6584a-7319-4c35-bab4-8871e138daa1 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:15.602139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:15.602753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:15.603031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:15.603302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:15.614014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:15.615483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:15.630129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:15.631202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:15.632288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:15.632288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:15.632288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:15.632288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 642/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:15.637407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:15.734095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535438a-2a66-402d-869c-23fc75156096 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:15.766648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:15.767245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:15.767522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:15.767793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:15.768423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:15.768719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:15.770102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:15.781351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 628/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 04:51:15.787043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:15.817060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:15.886524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745b858-55aa-4bb2-8971-057021003468 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:15.919573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:15.920190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:15.920467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:15.920737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:15.927232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:15.928662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:51:15.940134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:15.941174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:15.942329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 643/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:15.947069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:16.003864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:16.045428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569baa38-5ee3-43a6-bcfe-0d6853a329c2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:16.078155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:16.078743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:16.079021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:16.079292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:16.079971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:16.080272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:16.081683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd6edb-a92b-48e4-9ac3-55f63ef5e51c None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:16.093067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 629/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:51:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:16.098880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:16.187375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:16.195859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37193bdc-f3f2-4bdd-98fe-1995d4b43090 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:16.227670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7957b0-4669-4a8e-a8ee-e1fbc6548386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:16.228305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7957b0-4669-4a8e-a8ee-e1fbc6548386 None admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:16.228579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7957b0-4669-4a8e-a8ee-e1fbc6548386 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:16.228847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7957b0-4669-4a8e-a8ee-e1fbc6548386 None admin] PATH_INFO: `/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/fdf673c05e4443e8bce9b9d7371d1fd7/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:16.230675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7957b0-4669-4a8e-a8ee-e1fbc6548386 None admin] RBAC: Authorizing `identity:create_grant(project_id=308491fe36314e479f19ead78bb4b5a6, user_id=fdf673c05e4443e8bce9b9d7371d1fd7, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:16.232616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7957b0-4669-4a8e-a8ee-e1fbc6548386 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:16.244230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 644/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Sat Sep 24 04:51:16 2022] PUT /identity/v3/projects/308491fe36314e479f19ead78bb4b5a6/users/fdf673c05e4443e8bce9b9d7371d1fd7/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:51:18.371580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc883-d99d-4c32-97c7-cf32c2288d37 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:18.372168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc883-d99d-4c32-97c7-cf32c2288d37 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:18.372168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc883-d99d-4c32-97c7-cf32c2288d37 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:18.373471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 630/1274] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:18.381895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:18.382558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:18.383143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:18.411902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:18.413313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:18.478935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 645/1275] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:18.754481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc883-d99d-4c32-97c7-cf32c2288d37 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:18.852247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc883-d99d-4c32-97c7-cf32c2288d37 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:18.869625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:18.886802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:18.887419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:18.887694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:18.887963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:18.898280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 24 04:51:18.903042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:18.904148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:18.904998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 631/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Sat Sep 24 04:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:18.910007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:18.987831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:19.006176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b55ca-13b8-4f9d-b127-12dfa2d7543a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:19.038596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:19.039189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:19.039463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:19.039731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:19.040373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:19.040670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:19.042035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:19.049686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 646/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Sat Sep 24 04:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:19.055595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:19.153040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6234bf-78bc-4fac-93c3-c17d6ff8240d None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:19.185933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:19.186513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:19.186789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:19.187056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:19.197166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:19.198629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:19.212882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:19.214043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:19.215072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:19.215072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:19.215072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:19.215072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 632/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:19.219775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:19.315680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27c785-76c0-47e3-853c-4956b0f6f919 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:19.348120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:19.348859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:19.349143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:19.349414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:19.350077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:19.350375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:19.351721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:19.363681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 647/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:19.369327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:19.399745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:19.466630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b43bd-515e-49b9-a405-8b24e8492618 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:19.499018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:19.499628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:19.499904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:19.500173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:19.507211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:19.508631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 24 04:51:19.519761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:19.520864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:19.521859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 633/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:19.526465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:19.530164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:19.622288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb8844-c05c-481a-bf13-5abebfac748b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:19.654994 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:19.655595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:19.655871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:19.656148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:19.656863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:19.657163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:19.658614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:19.675644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 648/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Sat Sep 24 04:51:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:19.681427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:19.770653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:19.780737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b42ab2-2be7-4e02-884d-f9898e429620 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:19.812966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:19.813550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:19.813824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:19.814127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:19.878442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:19.879874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:19.966296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:19.976491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 634/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Sat Sep 24 04:51:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 24 04:51:19.982796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:20.042253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:20.080448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86865cfc-104f-483f-adbe-95450ef6b388 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:20.113404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:20.114045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:20.114326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:20.114599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:20.124863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:20.126316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 24 04:51:20.140724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:20.141839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 24 04:51:20.143129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 24 04:51:20.143129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 24 04:51:20.143129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:20.143129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 649/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:51:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:20.147497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:20.244133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994a06c-7659-4db7-8328-163150cef655 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:20.277627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:20.278259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:20.278536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:20.278831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:20.279463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:20.279758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:20.281114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:20.291903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 635/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Sat Sep 24 04:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:20.298182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:20.395530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1ae26-37ef-4896-8960-26c19451453b None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:20.408881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:20.428997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:20.429612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:20.429892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:20.430200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:20.436607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:20.438078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 24 04:51:20.448998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:20.450038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: ERROR keystone.server.flask.application Sep 24 04:51:20.451109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 650/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Sat Sep 24 04:51:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:20.455876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:20.504360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:20.553216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e8126-86ca-43b5-b0e7-591ea8fb5b28 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:20.584974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:20.585563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:20.585840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:20.586139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:20.586812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:20.587113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:20.588524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eaa364-d166-44a6-8539-b56b20689d56 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:20.600541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 636/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Sat Sep 24 04:51:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:20.606420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:20.703436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c4e2-037f-4574-bc2e-eeeb16df6112 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:20.736724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:20.737312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:20.737585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:20.737852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:20.738585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] RBAC: Adding query filter params (role.id=f3cd0a39225e41639953039b68ac4c88, scope.project.id=308491fe36314e479f19ead78bb4b5a6, user.id=fdf673c05e4443e8bce9b9d7371d1fd7) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:20.738888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:20.740322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3fce1-d45f-4908-9f71-d4e77bad0e3e None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:20.745820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 651/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Sat Sep 24 04:51:20 2022] GET /identity/v3/role_assignments?user.id=fdf673c05e4443e8bce9b9d7371d1fd7&scope.project.id=308491fe36314e479f19ead78bb4b5a6&role.id=f3cd0a39225e41639953039b68ac4c88 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:22.911157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067692d-3270-499e-9c80-ce0c5ec3d591 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:22.911674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067692d-3270-499e-9c80-ce0c5ec3d591 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:22.911674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067692d-3270-499e-9c80-ce0c5ec3d591 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:22.912934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 637/1288] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:22.918292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:22.918710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:22.918841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:22.937084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:22.938294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:22.966803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:23.002846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 652/1289] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:23.266447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067692d-3270-499e-9c80-ce0c5ec3d591 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:23.369135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067692d-3270-499e-9c80-ce0c5ec3d591 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:23.381199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:23.408166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:23.409268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:23.409570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:23.409944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:23.410841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:23.412910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:23.423454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:23.424682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:23.425584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 638/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:51:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:23.430764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:23.524079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8f208-6e5b-4cb0-a149-120d751711bc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:23.556656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:23.557248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:23.557525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:23.557795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:23.558445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:23.558743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:23.560120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c9a04-3029-40d8-8fd5-c1808e93f042 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:23.564737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 653/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Sat Sep 24 04:51:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:23.570447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:23.674397 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5661608-ce5f-4a8d-8625-d6fe689a1d06 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:23.706304 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:23.706897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:23.707169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:23.707438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] PATH_INFO: `/v3/services` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:23.708050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] RBAC: Adding query filter params (type=placement) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:23.708342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:23.709689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07708bb8-1bc1-49a9-b581-af978aab21ec None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:23.713912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 639/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Sat Sep 24 04:51:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:25.812422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e21feb-d724-4c06-8d56-c42c386d9278 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:25.812958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e21feb-d724-4c06-8d56-c42c386d9278 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:25.812958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e21feb-d724-4c06-8d56-c42c386d9278 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:25.814346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 654/1293] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:25.819346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f162-a29c-447a-a91f-3b00352e1dff None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:25.819731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f162-a29c-447a-a91f-3b00352e1dff None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:25.819890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30f162-a29c-447a-a91f-3b00352e1dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:25.838017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5c30f162-a29c-447a-a91f-3b00352e1dff None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:25.839190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5c30f162-a29c-447a-a91f-3b00352e1dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:25.867191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:25.904908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 640/1294] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:26.171002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e21feb-d724-4c06-8d56-c42c386d9278 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:26.201479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:26.268895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e21feb-d724-4c06-8d56-c42c386d9278 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:26.301094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b09ef-8378-4a2f-a646-fa1ae1662413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:26.301686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b09ef-8378-4a2f-a646-fa1ae1662413 None admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:26.301964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b09ef-8378-4a2f-a646-fa1ae1662413 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:26.302266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b09ef-8378-4a2f-a646-fa1ae1662413 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:26.302916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b09ef-8378-4a2f-a646-fa1ae1662413 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:26.304261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b09ef-8378-4a2f-a646-fa1ae1662413 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:26.319286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 655/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Sat Sep 24 04:51:26 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 04:51:28.430971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c246c-6dba-424d-85d5-2089c204f99d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:28.431496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c246c-6dba-424d-85d5-2089c204f99d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:28.431496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c246c-6dba-424d-85d5-2089c204f99d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:28.432851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 641/1296] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:28.438122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:28.438519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:28.438668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:28.456729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:28.457889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:28.526632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 656/1297] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:28.830866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c246c-6dba-424d-85d5-2089c204f99d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:28.929391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c246c-6dba-424d-85d5-2089c204f99d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:28.963701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:28.964370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:28.964645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:28.964912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:28.965481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:28.966952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:28.975492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:28.976643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:28.977665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 642/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:51:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:28.982908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:29.001035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:29.086838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345fddb-d1e0-4400-b12a-5fa11273bc21 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:29.118962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:29.119558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:29.119841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:29.120111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:29.120729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:29.121023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:29.122401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:29.127941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 657/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Sat Sep 24 04:51:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:29.133875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:29.164867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:29.230735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923d6d6-c477-4ca8-ba57-1d9c401f059e None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:29.263552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:29.264143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:29.264418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:29.264686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:29.265368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] RBAC: Adding query filter params (interface=public, service_id=5a066a2b929248e2b2d79585ea206988, region_id=RegionOne) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:29.265667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:29.267047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb7924-35b8-4034-8a5e-d20ae59674ef None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:29.271847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 643/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Sat Sep 24 04:51:29 2022] GET /identity/v3/endpoints?service_id=5a066a2b929248e2b2d79585ea206988&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:29.277536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:29.361270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:29.383195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d9c9f-ab89-486d-adcd-c630c62ad363 None admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:29.415413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658f442-3158-46e9-b762-fd7d1fdb591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:29.416000 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658f442-3158-46e9-b762-fd7d1fdb591c None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:29.416273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658f442-3158-46e9-b762-fd7d1fdb591c None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:29.416539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658f442-3158-46e9-b762-fd7d1fdb591c None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:29.417138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658f442-3158-46e9-b762-fd7d1fdb591c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:29.418543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658f442-3158-46e9-b762-fd7d1fdb591c None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:29.425636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 658/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Sat Sep 24 04:51:29 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:51:31.531208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa6d5-b32e-4107-af3f-970db5c9ae2e None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:31.531764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa6d5-b32e-4107-af3f-970db5c9ae2e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:31.531764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfa6d5-b32e-4107-af3f-970db5c9ae2e None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:31.533063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 644/1302] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:51:31.538323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:31.538718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:31.538882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:31.556565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:51:31.557797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:51:31.625186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 659/1303] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:51:31 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 24 04:51:31.895369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfa6d5-b32e-4107-af3f-970db5c9ae2e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:31.987937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:31.994816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfa6d5-b32e-4107-af3f-970db5c9ae2e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:32.027560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:32.028173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:32.028449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:32.028719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:32.029934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:32.031308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 24 04:51:32.039807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 24 04:51:32.040972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.get_or_create( Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application with Lock( Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self._enter() Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application return self.creator() Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application created_value = creator( Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: ERROR keystone.server.flask.application Sep 24 04:51:32.042041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 645/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Sat Sep 24 04:51:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 24 04:51:32.047103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:32.118670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:32.141130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075d140-1f66-4317-a093-8d46a7667c6e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:32.173298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:32.173833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:32.174138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:32.174410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] PATH_INFO: `/v3/services` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:32.175031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:51:32.175327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:32.176692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd92f0-334c-4e40-b389-360e8ef33fc0 None admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:32.181630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 660/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Sat Sep 24 04:51:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:51:32.187896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:51:32.284583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be001431-cf40-4157-9c0d-eed734af06a2 None admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:51:32.317707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935a19e-6ad6-4fff-bd53-bc0055fab03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:51:32.318321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935a19e-6ad6-4fff-bd53-bc0055fab03b None admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:51:32.318607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935a19e-6ad6-4fff-bd53-bc0055fab03b None admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:51:32.318883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935a19e-6ad6-4fff-bd53-bc0055fab03b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:51:32.319545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935a19e-6ad6-4fff-bd53-bc0055fab03b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:51:32.320890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935a19e-6ad6-4fff-bd53-bc0055fab03b None admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:51:32.339238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 646/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:51:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:51:42.178871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=30 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:51:42.335106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:36.022340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dbf08-cc7c-4ff6-b33d-b4f4871f2005 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:36.023209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dbf08-cc7c-4ff6-b33d-b4f4871f2005 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:36.023959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dbf08-cc7c-4ff6-b33d-b4f4871f2005 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:36.031205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 661/1307] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:36 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:36.032679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a09b0-1ad5-4926-bbc3-9126b565c629 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:36.033114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a09b0-1ad5-4926-bbc3-9126b565c629 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:36.033264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a09b0-1ad5-4926-bbc3-9126b565c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:36.183548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-aa4a09b0-1ad5-4926-bbc3-9126b565c629 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:36.184777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-aa4a09b0-1ad5-4926-bbc3-9126b565c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:36.401457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 647/1308] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:36.693140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d13e29-7548-45c7-aae9-01e889f4d6d8 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:36.693769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d13e29-7548-45c7-aae9-01e889f4d6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:36.694285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d13e29-7548-45c7-aae9-01e889f4d6d8 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:36.695869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 662/1309] 10.222.0.50 () {54 vars in 964 bytes} [Sat Sep 24 04:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:36.703064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e78b5-d710-4084-be1e-03c72e4c41b7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:36.703393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e78b5-d710-4084-be1e-03c72e4c41b7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:36.703614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e78b5-d710-4084-be1e-03c72e4c41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:36.818293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-830e78b5-d710-4084-be1e-03c72e4c41b7 None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:36.819550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-830e78b5-d710-4084-be1e-03c72e4c41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:36.852765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:36.919672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 648/1310] 10.222.0.50 () {58 vars in 1061 bytes} [Sat Sep 24 04:52:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:36.934970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d13e29-7548-45c7-aae9-01e889f4d6d8 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:36.937040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e4d13e29-7548-45c7-aae9-01e889f4d6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:37.073869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d13e29-7548-45c7-aae9-01e889f4d6d8 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:37.164396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:37.165424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:37.165930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:37.166428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:37.168310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:37.239344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:37.305216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:37.307172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad78db-c189-40ec-a9c6-a2c182b7b135 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:37.329357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 663/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:39.837723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d37c8-ebff-40d1-b746-e51782db6fde None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:39.838289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d37c8-ebff-40d1-b746-e51782db6fde None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:39.838413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d37c8-ebff-40d1-b746-e51782db6fde None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:39.839636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 649/1312] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:39.845333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a548908-94c4-4ca7-b5d3-ede9efa852a8 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:39.845755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a548908-94c4-4ca7-b5d3-ede9efa852a8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:39.845868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a548908-94c4-4ca7-b5d3-ede9efa852a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:39.925988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3a548908-94c4-4ca7-b5d3-ede9efa852a8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:39.927220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3a548908-94c4-4ca7-b5d3-ede9efa852a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:40.020499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 664/1313] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:40.302475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d37c8-ebff-40d1-b746-e51782db6fde None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:40.304013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-155d37c8-ebff-40d1-b746-e51782db6fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:40.404862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d37c8-ebff-40d1-b746-e51782db6fde None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:40.475038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:40.477347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:40.478151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:40.478475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:40.478958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:40.480861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:40.588061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:40.589408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c708845-4cbb-4814-8148-ad0f8f29f790 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:40.607629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 650/1314] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:43.040857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05321d5c-640c-48f8-b8e2-96157e18a0c5 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:43.041394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05321d5c-640c-48f8-b8e2-96157e18a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:43.041394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05321d5c-640c-48f8-b8e2-96157e18a0c5 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:43.042790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 665/1315] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:43.048709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b298a-100e-4c70-92e2-15c2c4214e66 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:43.049138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b298a-100e-4c70-92e2-15c2c4214e66 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:43.049232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b298a-100e-4c70-92e2-15c2c4214e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:43.115548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f57b298a-100e-4c70-92e2-15c2c4214e66 None None] MFA Rules not processed for user `c6c71c8a128a4b5f93b4e8d9cf8f37da`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:43.116747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f57b298a-100e-4c70-92e2-15c2c4214e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:43.167692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:43.211953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 651/1316] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:43.507709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05321d5c-640c-48f8-b8e2-96157e18a0c5 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:43.509306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-05321d5c-640c-48f8-b8e2-96157e18a0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:43.520234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:43.620224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05321d5c-640c-48f8-b8e2-96157e18a0c5 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:43.700601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:43.701213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:43.701492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:43.701764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:43.703421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:43.764255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:43.812854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:43.817003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46373bf2-63bf-45e9-a2bd-428b22072603 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:43.836008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 666/1317] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:46.578595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31febc-e421-4ce6-b325-0eb2b470eb92 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:46.579125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31febc-e421-4ce6-b325-0eb2b470eb92 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:46.579125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31febc-e421-4ce6-b325-0eb2b470eb92 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:46.580468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 652/1318] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:46.585840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11abc2-15cd-4f13-ae57-19cdbce03f10 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:46.586246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11abc2-15cd-4f13-ae57-19cdbce03f10 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:46.586428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11abc2-15cd-4f13-ae57-19cdbce03f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:46.651758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-be11abc2-15cd-4f13-ae57-19cdbce03f10 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:46.653017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-be11abc2-15cd-4f13-ae57-19cdbce03f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:46.747152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 667/1319] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:47.027518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31febc-e421-4ce6-b325-0eb2b470eb92 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:47.029062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1c31febc-e421-4ce6-b325-0eb2b470eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:47.130542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c31febc-e421-4ce6-b325-0eb2b470eb92 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:47.201439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:47.202048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:47.202324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:47.203109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:47.204676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:47.211393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:47.312273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:47.313623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e82dc3-8e72-454d-99c1-89dc7e4fb29a service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:47.331437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 653/1320] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:50.104937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b28e0-50f4-47d9-ae74-d0439d9d0081 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:50.105454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b28e0-50f4-47d9-ae74-d0439d9d0081 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:50.105545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b28e0-50f4-47d9-ae74-d0439d9d0081 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:50.106915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 668/1321] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:50.112824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126afd-a738-41c7-b9a7-edbb383da2d4 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:50.113271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126afd-a738-41c7-b9a7-edbb383da2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:50.113499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c126afd-a738-41c7-b9a7-edbb383da2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:50.186670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7c126afd-a738-41c7-b9a7-edbb383da2d4 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:50.188656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7c126afd-a738-41c7-b9a7-edbb383da2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:50.248965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:50.291519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 654/1322] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:52:50.577046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b28e0-50f4-47d9-ae74-d0439d9d0081 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:50.579282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-415b28e0-50f4-47d9-ae74-d0439d9d0081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:50.599144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:50.684742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b28e0-50f4-47d9-ae74-d0439d9d0081 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:50.759415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:50.760020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:50.760298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:50.760567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:50.762187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:50.833190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:50.872894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:50.874403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2899479f-b829-4334-b6e9-cd6edafd1b85 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:50.895223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 669/1323] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:53.083497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452fb19-4602-424f-9268-2184d0b1215a None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:53.084090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452fb19-4602-424f-9268-2184d0b1215a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:53.084090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d452fb19-4602-424f-9268-2184d0b1215a None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:53.085399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 655/1324] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:53.091297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d505f-38b2-44b1-9736-0dac298afd8c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:53.091862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d505f-38b2-44b1-9736-0dac298afd8c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:53.091862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d505f-38b2-44b1-9736-0dac298afd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:53.159005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4b7d505f-38b2-44b1-9736-0dac298afd8c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:53.160214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4b7d505f-38b2-44b1-9736-0dac298afd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:53.256915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 670/1325] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:53.540648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452fb19-4602-424f-9268-2184d0b1215a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:53.542205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d452fb19-4602-424f-9268-2184d0b1215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:53.670987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d452fb19-4602-424f-9268-2184d0b1215a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:53.749701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:53.750451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:53.750820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:53.751165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:53.752849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:53.782352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:53.871047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:53.872427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d2878-8855-4761-9910-b27c42b0f463 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:53.894174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 656/1326] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:56.108872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d27276-952b-4457-9d95-16aeef477654 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:56.109418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d27276-952b-4457-9d95-16aeef477654 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:56.109418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d27276-952b-4457-9d95-16aeef477654 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:56.110803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 671/1327] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:56.116043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ea0d-6fb4-4d5f-9a29-4c3575b33302 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:56.116421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ea0d-6fb4-4d5f-9a29-4c3575b33302 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:56.116588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5480ea0d-6fb4-4d5f-9a29-4c3575b33302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:56.182160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5480ea0d-6fb4-4d5f-9a29-4c3575b33302 None None] MFA Rules not processed for user `c6c71c8a128a4b5f93b4e8d9cf8f37da`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:56.183283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5480ea0d-6fb4-4d5f-9a29-4c3575b33302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:56.246414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:56.277990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 657/1328] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:56.587323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bce02-fe65-4cae-9874-9918f7db1cee None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:56.587603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bce02-fe65-4cae-9874-9918f7db1cee None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:56.587881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bce02-fe65-4cae-9874-9918f7db1cee None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:56.589289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 672/1329] 10.222.0.50 () {54 vars in 964 bytes} [Sat Sep 24 04:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:56.596213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14aa75c-3364-4e8d-8f44-076696d9461a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:56.596503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14aa75c-3364-4e8d-8f44-076696d9461a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:56.596780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14aa75c-3364-4e8d-8f44-076696d9461a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:56.683773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b14aa75c-3364-4e8d-8f44-076696d9461a None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:56.685007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b14aa75c-3364-4e8d-8f44-076696d9461a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:56.762543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 658/1330] 10.222.0.50 () {58 vars in 1061 bytes} [Sat Sep 24 04:52:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:52:56.772379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bce02-fe65-4cae-9874-9918f7db1cee None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:52:56.773908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0f3bce02-fe65-4cae-9874-9918f7db1cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:56.805103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:56.877501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bce02-fe65-4cae-9874-9918f7db1cee None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:52:56.950441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:52:56.951047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:56.951325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:56.951597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:56.953174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:52:57.029213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:52:57.061031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:52:57.062998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c570c-5cde-48a5-bdcc-d8f26553d433 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:52:57.080574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 673/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:52:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:52:59.848249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6ec0b-485c-4211-9388-35e2574126a3 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:59.848781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6ec0b-485c-4211-9388-35e2574126a3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:59.848781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6ec0b-485c-4211-9388-35e2574126a3 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:59.850052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 659/1332] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:52:59.855490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f515-973f-473f-867b-95bf7228d571 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:52:59.855888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f515-973f-473f-867b-95bf7228d571 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:52:59.856023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565f515-973f-473f-867b-95bf7228d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:52:59.921101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2565f515-973f-473f-867b-95bf7228d571 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:52:59.922364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2565f515-973f-473f-867b-95bf7228d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:00.014696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 674/1333] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:00.297425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6ec0b-485c-4211-9388-35e2574126a3 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:00.299017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c2c6ec0b-485c-4211-9388-35e2574126a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:00.367114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:00.401013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6ec0b-485c-4211-9388-35e2574126a3 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:00.472908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:00.473515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:00.473788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:00.474079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:00.475650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:00.581573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:00.582948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1e120-deab-4a78-b7a3-01d8d3054abb service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:00.592550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:00.601082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 660/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 04:53:02.798439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153226-913d-4b67-8e98-4d1c0460d075 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:02.799016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153226-913d-4b67-8e98-4d1c0460d075 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:02.799016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91153226-913d-4b67-8e98-4d1c0460d075 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:02.800353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 675/1335] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:02.805610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356bdcf-cee7-46dc-905b-7f170e15587d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:02.806065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356bdcf-cee7-46dc-905b-7f170e15587d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:02.806176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356bdcf-cee7-46dc-905b-7f170e15587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:02.870735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2356bdcf-cee7-46dc-905b-7f170e15587d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:02.871935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2356bdcf-cee7-46dc-905b-7f170e15587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:02.967115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 661/1336] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:03.253501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91153226-913d-4b67-8e98-4d1c0460d075 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:03.255082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-91153226-913d-4b67-8e98-4d1c0460d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:03.294496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:03.356491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91153226-913d-4b67-8e98-4d1c0460d075 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:03.427926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:03.428532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:03.428810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:03.429080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:03.430710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:03.513490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:03.538895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:03.540334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61eadc-5c4b-4f46-9144-2e01e4a92ca2 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:03.559335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 676/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 04:53:05.999510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73bac-3a36-4da6-ac28-10b001317e40 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:06.000065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73bac-3a36-4da6-ac28-10b001317e40 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:06.000065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf73bac-3a36-4da6-ac28-10b001317e40 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:06.001411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 662/1338] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:06.006717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f54c86-a2dd-4feb-92d1-a44b855f84ff None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:06.007170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f54c86-a2dd-4feb-92d1-a44b855f84ff None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:06.007266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f54c86-a2dd-4feb-92d1-a44b855f84ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:06.071569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-97f54c86-a2dd-4feb-92d1-a44b855f84ff None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:06.072754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-97f54c86-a2dd-4feb-92d1-a44b855f84ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:06.167411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 677/1339] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:06.444804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73bac-3a36-4da6-ac28-10b001317e40 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:06.446367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ddf73bac-3a36-4da6-ac28-10b001317e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:06.521523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:06.548877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf73bac-3a36-4da6-ac28-10b001317e40 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:06.627200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:06.627723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:06.627995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:06.628264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:06.629845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:06.736703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:06.738174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad467bd-a264-4191-bfa8-d0738b8f3265 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:06.756675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 663/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:10.815918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e032e6-3757-43fb-9d74-8bdaa7a72881 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:10.816508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e032e6-3757-43fb-9d74-8bdaa7a72881 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:10.816508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e032e6-3757-43fb-9d74-8bdaa7a72881 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:10.817971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 678/1341] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:10.823136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd698-4da8-4f11-aa87-de5e982ee0e1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:10.823533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd698-4da8-4f11-aa87-de5e982ee0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:10.823741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fd698-4da8-4f11-aa87-de5e982ee0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:10.829763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:10.894831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-459fd698-4da8-4f11-aa87-de5e982ee0e1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:10.895943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-459fd698-4da8-4f11-aa87-de5e982ee0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:10.994883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 664/1342] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:53:11.286795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e032e6-3757-43fb-9d74-8bdaa7a72881 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:11.288367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-76e032e6-3757-43fb-9d74-8bdaa7a72881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:11.337102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:11.392762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e032e6-3757-43fb-9d74-8bdaa7a72881 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:11.464028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:11.464645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:11.464919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:11.465183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:11.466869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:11.565448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:11.581465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:11.582934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf53e7-6f43-4b3b-8bf0-e003fc268181 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:11.602538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 679/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:14.379147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeccf1-4b25-4857-818f-b3b455cf2c51 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:14.379672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeccf1-4b25-4857-818f-b3b455cf2c51 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:14.379672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adeccf1-4b25-4857-818f-b3b455cf2c51 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:14.380942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 665/1344] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:14.386191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d45b9-3096-47e9-a20c-37c95188e343 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:14.386613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d45b9-3096-47e9-a20c-37c95188e343 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:14.386742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d45b9-3096-47e9-a20c-37c95188e343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:14.452326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-608d45b9-3096-47e9-a20c-37c95188e343 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:14.454166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-608d45b9-3096-47e9-a20c-37c95188e343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:14.547790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 680/1345] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:14.825948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adeccf1-4b25-4857-818f-b3b455cf2c51 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:14.827586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4adeccf1-4b25-4857-818f-b3b455cf2c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:14.919504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:14.938681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adeccf1-4b25-4857-818f-b3b455cf2c51 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:15.014761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:15.015362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:15.015635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:15.016523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:15.018135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:15.130188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:15.131640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f336eb9-8482-4ac1-83b0-c3e4e587e958 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:15.154316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 666/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:19.270882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fecdb6-3959-4380-9c18-27e83f083927 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:19.271350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fecdb6-3959-4380-9c18-27e83f083927 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:19.271442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fecdb6-3959-4380-9c18-27e83f083927 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:19.272960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 681/1347] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:19.278155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf37160-aed8-4104-80df-7054152d0b7f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:19.278600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf37160-aed8-4104-80df-7054152d0b7f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:19.278900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf37160-aed8-4104-80df-7054152d0b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:19.295307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:19.345477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-daf37160-aed8-4104-80df-7054152d0b7f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:19.346689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-daf37160-aed8-4104-80df-7054152d0b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:19.440942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 667/1348] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:19.720494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fecdb6-3959-4380-9c18-27e83f083927 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:19.722062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b3fecdb6-3959-4380-9c18-27e83f083927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:19.779923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:19.825988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fecdb6-3959-4380-9c18-27e83f083927 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:19.903892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:19.904495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:19.904769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:19.905040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:19.906725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:20.012851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:20.017217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:20.018957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0b6b1-abaa-4df2-ade3-b88b7665c2e5 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:20.037127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 682/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:24.904734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749ad84-a875-4da4-886a-f0a4c8ed2d88 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:24.905244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749ad84-a875-4da4-886a-f0a4c8ed2d88 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:24.905340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749ad84-a875-4da4-886a-f0a4c8ed2d88 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:24.907012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 668/1350] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 04:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:24.912476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c812f-d6c1-475b-96f1-ade48b6461f9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:24.912888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c812f-d6c1-475b-96f1-ade48b6461f9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:24.913031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c812f-d6c1-475b-96f1-ade48b6461f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:25.001178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d55c812f-d6c1-475b-96f1-ade48b6461f9 None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:25.002451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d55c812f-d6c1-475b-96f1-ade48b6461f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:25.176535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 683/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 04:53:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:53:29.373808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c3ff-cdcc-40aa-8938-de8cb1b74b3f None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:29.374702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c3ff-cdcc-40aa-8938-de8cb1b74b3f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:29.375131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79c3ff-cdcc-40aa-8938-de8cb1b74b3f None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:29.377362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 669/1352] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 04:53:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:29.382730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559453f5-30b9-4948-97fc-7a138b5c074a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:29.383189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559453f5-30b9-4948-97fc-7a138b5c074a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:29.383438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559453f5-30b9-4948-97fc-7a138b5c074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:29.439068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:29.439361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=59 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:29.465113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-559453f5-30b9-4948-97fc-7a138b5c074a None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:29.466437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-559453f5-30b9-4948-97fc-7a138b5c074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:29.576904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 684/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 04:53:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:30.834154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934250c-3fb4-4ef2-b72f-918ca3e5c624 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:30.834698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934250c-3fb4-4ef2-b72f-918ca3e5c624 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:30.834698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934250c-3fb4-4ef2-b72f-918ca3e5c624 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:30.836061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 670/1354] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 04:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:30.841163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efcfaa-c52e-464d-af2c-b1c1c1dfe093 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:30.841587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efcfaa-c52e-464d-af2c-b1c1c1dfe093 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:30.841702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efcfaa-c52e-464d-af2c-b1c1c1dfe093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:30.918910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-37efcfaa-c52e-464d-af2c-b1c1c1dfe093 None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:30.920037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-37efcfaa-c52e-464d-af2c-b1c1c1dfe093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:30.961934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:31.029514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 685/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 04:53:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:34.646815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1fec-0a07-403e-8043-f93e29050d6d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:34.647670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1fec-0a07-403e-8043-f93e29050d6d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:34.648482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1fec-0a07-403e-8043-f93e29050d6d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:34.654278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 671/1356] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:34.655586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e75b-0b6d-456f-8289-c6b4fede781f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:34.656011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e75b-0b6d-456f-8289-c6b4fede781f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:34.656143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e75b-0b6d-456f-8289-c6b4fede781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:34.734751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f6a1e75b-0b6d-456f-8289-c6b4fede781f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:34.735990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f6a1e75b-0b6d-456f-8289-c6b4fede781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:34.842037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 686/1357] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:35.148045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84ed7e-0840-442f-a4a0-bce11733a487 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:35.148428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84ed7e-0840-442f-a4a0-bce11733a487 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:35.148795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84ed7e-0840-442f-a4a0-bce11733a487 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:35.150249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 672/1358] 10.222.0.50 () {54 vars in 961 bytes} [Sat Sep 24 04:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:35.163563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a155ddc-3b80-4570-9410-f3c3924dcd49 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:35.164030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a155ddc-3b80-4570-9410-f3c3924dcd49 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:35.164391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a155ddc-3b80-4570-9410-f3c3924dcd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:35.184091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:35.252342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0a155ddc-3b80-4570-9410-f3c3924dcd49 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:35.253682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0a155ddc-3b80-4570-9410-f3c3924dcd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:35.378250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 687/1359] 10.222.0.50 () {58 vars in 1058 bytes} [Sat Sep 24 04:53:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:35.384713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84ed7e-0840-442f-a4a0-bce11733a487 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:35.386554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6c84ed7e-0840-442f-a4a0-bce11733a487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:35.541611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84ed7e-0840-442f-a4a0-bce11733a487 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:35.624556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:35.663731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:35.664448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:35.664819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:35.665182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:35.666951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:35.764982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:35.766660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c38f82-a4f4-4999-9496-ad2bfddcdc4e service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:35.787074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 673/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:38.177525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4c13-2bef-4575-9aa4-47a4489d93f9 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:38.178227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4c13-2bef-4575-9aa4-47a4489d93f9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:38.178227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4c13-2bef-4575-9aa4-47a4489d93f9 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:38.179279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 688/1361] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:38.184758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286172e-9597-4125-80c0-88508bffcb63 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:38.185586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286172e-9597-4125-80c0-88508bffcb63 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:38.185865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286172e-9597-4125-80c0-88508bffcb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:38.257558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-1286172e-9597-4125-80c0-88508bffcb63 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:38.259273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1286172e-9597-4125-80c0-88508bffcb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:38.298077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:38.365290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 674/1362] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:38.668899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4c13-2bef-4575-9aa4-47a4489d93f9 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:38.670908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-869a4c13-2bef-4575-9aa4-47a4489d93f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:38.750635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:38.864856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4c13-2bef-4575-9aa4-47a4489d93f9 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:38.999462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:39.000068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:39.000346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:39.000619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:39.002842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:39.026294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:39.095556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:39.097202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a5705-cffa-43f9-9198-23a6cc2cc0cb service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:39.116921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 689/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:41.644818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362559-a702-4664-900c-e6c07410a5d6 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:41.645584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362559-a702-4664-900c-e6c07410a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:41.645687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362559-a702-4664-900c-e6c07410a5d6 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:41.649508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 675/1364] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:41.653117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779e24-4c9c-47a5-9fef-4874bf23e88c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:41.653117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779e24-4c9c-47a5-9fef-4874bf23e88c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:41.653604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779e24-4c9c-47a5-9fef-4874bf23e88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:41.723437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-28779e24-4c9c-47a5-9fef-4874bf23e88c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:41.724651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-28779e24-4c9c-47a5-9fef-4874bf23e88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:41.824561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:41.838279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 690/1365] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:42.115953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716fe87-9dd1-44fb-aefd-f3fe9a85847a None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:42.116425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716fe87-9dd1-44fb-aefd-f3fe9a85847a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:42.116699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716fe87-9dd1-44fb-aefd-f3fe9a85847a None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:42.118039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 676/1366] 10.222.0.50 () {54 vars in 961 bytes} [Sat Sep 24 04:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:42.123471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad55b-508e-415c-8b54-7a8e7d837e3e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:42.123751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad55b-508e-415c-8b54-7a8e7d837e3e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:42.124019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad55b-508e-415c-8b54-7a8e7d837e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:42.217267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-279ad55b-508e-415c-8b54-7a8e7d837e3e None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:42.219074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-279ad55b-508e-415c-8b54-7a8e7d837e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:42.349852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 691/1367] 10.222.0.50 () {58 vars in 1058 bytes} [Sat Sep 24 04:53:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:42.357870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716fe87-9dd1-44fb-aefd-f3fe9a85847a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:42.359564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3716fe87-9dd1-44fb-aefd-f3fe9a85847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:42.515246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716fe87-9dd1-44fb-aefd-f3fe9a85847a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:42.545380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:42.666757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:42.667497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:42.667905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:42.668292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:42.670021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:42.771964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:42.773857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5701dd9-b762-466e-beb8-da0969383662 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:42.793289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 677/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:42.834433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30423265-d345-4861-a836-d43687cada86 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:42.834965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30423265-d345-4861-a836-d43687cada86 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:42.835228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30423265-d345-4861-a836-d43687cada86 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:42.836923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 692/1369] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:42.842621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f6fc0-b035-4db5-adc6-8b18091062d2 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:42.842933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f6fc0-b035-4db5-adc6-8b18091062d2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:42.843235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f6fc0-b035-4db5-adc6-8b18091062d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:42.879401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:42.937976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-295f6fc0-b035-4db5-adc6-8b18091062d2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:42.939258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-295f6fc0-b035-4db5-adc6-8b18091062d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:43.058181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 678/1370] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:43.414390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c249d9-7499-4ff6-bda4-7cdd2d5154a3 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:43.414809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c249d9-7499-4ff6-bda4-7cdd2d5154a3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:43.415192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c249d9-7499-4ff6-bda4-7cdd2d5154a3 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:43.416922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 693/1371] 10.222.0.50 () {54 vars in 963 bytes} [Sat Sep 24 04:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:43.421674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c94f1-6d14-44ad-bfc4-86a6628cf079 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:43.422093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c94f1-6d14-44ad-bfc4-86a6628cf079 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:43.422463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c94f1-6d14-44ad-bfc4-86a6628cf079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:43.505821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2b3c94f1-6d14-44ad-bfc4-86a6628cf079 None None] MFA Rules not processed for user `3a5e44f909a94c74990b2e80f1da5169`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:43.507049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2b3c94f1-6d14-44ad-bfc4-86a6628cf079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:43.556946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:43.590636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 679/1372] 10.222.0.50 () {58 vars in 1060 bytes} [Sat Sep 24 04:53:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:43.598625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c249d9-7499-4ff6-bda4-7cdd2d5154a3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:43.600183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c4c249d9-7499-4ff6-bda4-7cdd2d5154a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:43.665144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:43.726957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c249d9-7499-4ff6-bda4-7cdd2d5154a3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:43.818550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:43.819160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:43.819494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:43.819683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:43.821294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:43.929005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:43.944139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:43.944139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383b57c-871d-4593-bbd1-2c33a50c613e service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:43.963583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 694/1373] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:45.095383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7110742-1b7f-4fc7-8f20-1e30aa3c2536 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:45.095954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7110742-1b7f-4fc7-8f20-1e30aa3c2536 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:45.095954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7110742-1b7f-4fc7-8f20-1e30aa3c2536 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:45.097269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 680/1374] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:45.102614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6356-df78-455c-acd2-a2f3dd241bcb None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:45.103002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6356-df78-455c-acd2-a2f3dd241bcb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:45.103159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6356-df78-455c-acd2-a2f3dd241bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:45.171086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-08ca6356-df78-455c-acd2-a2f3dd241bcb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:45.172381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-08ca6356-df78-455c-acd2-a2f3dd241bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:45.269250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 695/1375] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:45.562526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7110742-1b7f-4fc7-8f20-1e30aa3c2536 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:45.564111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c7110742-1b7f-4fc7-8f20-1e30aa3c2536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:45.724532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7110742-1b7f-4fc7-8f20-1e30aa3c2536 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:45.778247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:45.844349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:45.845677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:45.845830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:45.846210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:45.848642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:45.946541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:45.948165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c256550-451f-4245-90a2-284a85816c46 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:45.967652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 681/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 04:53:48.180203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a55a4-4d88-4ffe-b8e4-56ffb1e958bc None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:48.181167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a55a4-4d88-4ffe-b8e4-56ffb1e958bc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:48.181560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a55a4-4d88-4ffe-b8e4-56ffb1e958bc None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:48.183121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 696/1377] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:48.188919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdcab4-fda9-4a2d-9a9f-f1b62545a728 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:48.189350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdcab4-fda9-4a2d-9a9f-f1b62545a728 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:48.189461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdcab4-fda9-4a2d-9a9f-f1b62545a728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:48.268592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-62bdcab4-fda9-4a2d-9a9f-f1b62545a728 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:48.269795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-62bdcab4-fda9-4a2d-9a9f-f1b62545a728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:48.278954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:48.375593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 682/1378] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:48.667524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a55a4-4d88-4ffe-b8e4-56ffb1e958bc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:48.669594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9d7a55a4-4d88-4ffe-b8e4-56ffb1e958bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:48.733615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:48.821923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a55a4-4d88-4ffe-b8e4-56ffb1e958bc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:48.939260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:48.940211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:48.940577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:48.940939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:48.942619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:48.962360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:49.037700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:49.039474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734c4a9-5bb1-41db-9252-7e2e447768b8 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:49.059773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 697/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:51.245296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39471361-6408-4871-9329-1f778feb2e85 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:51.245780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39471361-6408-4871-9329-1f778feb2e85 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:51.245884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39471361-6408-4871-9329-1f778feb2e85 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:51.247328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 683/1380] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:51.252410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa2a4-e981-4a90-8243-86d0b1f0a735 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:51.252809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa2a4-e981-4a90-8243-86d0b1f0a735 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:51.252966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa2a4-e981-4a90-8243-86d0b1f0a735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:51.320675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-340fa2a4-e981-4a90-8243-86d0b1f0a735 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:51.321925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-340fa2a4-e981-4a90-8243-86d0b1f0a735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:51.441009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:51.458436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 698/1381] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:51.736386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39471361-6408-4871-9329-1f778feb2e85 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:51.738432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-39471361-6408-4871-9329-1f778feb2e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:51.901099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:51.911899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39471361-6408-4871-9329-1f778feb2e85 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:52.033979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:52.034626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:52.034920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:52.035186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:52.036836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:52.129530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:52.131194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf2917-cb0b-4bf4-b09d-b114a3bcbb6c service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:52.142689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:52.151696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 684/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:54.596794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced2420-f7f4-4123-843d-f4cf7a1f4819 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:54.597350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced2420-f7f4-4123-843d-f4cf7a1f4819 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:54.597350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced2420-f7f4-4123-843d-f4cf7a1f4819 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:54.598780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 699/1383] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:54.604230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbfe0-9de7-410e-9134-4f72a5fbb23c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:54.604642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbfe0-9de7-410e-9134-4f72a5fbb23c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:54.604804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cbfe0-9de7-410e-9134-4f72a5fbb23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:54.677635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-720cbfe0-9de7-410e-9134-4f72a5fbb23c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:54.678956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-720cbfe0-9de7-410e-9134-4f72a5fbb23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:54.792856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 685/1384] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:55.083454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced2420-f7f4-4123-843d-f4cf7a1f4819 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:55.084995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eced2420-f7f4-4123-843d-f4cf7a1f4819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:55.246368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced2420-f7f4-4123-843d-f4cf7a1f4819 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:55.316609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:55.365572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:55.366124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:55.366406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:55.366707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:55.368383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:55.460450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:55.462080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7cc53-7781-44c8-8448-f20f8044b97f service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:55.479745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 700/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:57.712754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced421d-114f-4302-bb9d-9ca9ed080f90 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:57.713318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced421d-114f-4302-bb9d-9ca9ed080f90 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:57.713318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced421d-114f-4302-bb9d-9ca9ed080f90 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:57.714721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 686/1386] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:57.720225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016189ec-5407-48e8-b9e2-47f2c215ec13 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:57.720661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016189ec-5407-48e8-b9e2-47f2c215ec13 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:57.720896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016189ec-5407-48e8-b9e2-47f2c215ec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:57.810692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-016189ec-5407-48e8-b9e2-47f2c215ec13 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:57.811962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-016189ec-5407-48e8-b9e2-47f2c215ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:57.847640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:57.912507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 701/1387] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:58.201918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced421d-114f-4302-bb9d-9ca9ed080f90 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:58.203486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5ced421d-114f-4302-bb9d-9ca9ed080f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:58.283015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:58.356150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced421d-114f-4302-bb9d-9ca9ed080f90 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:53:58.482386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:53:58.482991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:58.483265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:58.483529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:58.485645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:58.524006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:58.574363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:53:58.575939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af5bcd-c233-431e-ad84-704fb3f44345 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:53:58.593571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 687/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:53:59.243302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dc1eb-ac84-4892-b4b7-e23d6ec48a4f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:59.243838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dc1eb-ac84-4892-b4b7-e23d6ec48a4f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:59.243838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dc1eb-ac84-4892-b4b7-e23d6ec48a4f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:59.245100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 702/1389] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:59.250586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ada2-3e48-4b8b-baba-354f7354fa0d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:59.251030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ada2-3e48-4b8b-baba-354f7354fa0d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:59.251120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ada2-3e48-4b8b-baba-354f7354fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:59.317846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-1b08ada2-3e48-4b8b-baba-354f7354fa0d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:59.319122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1b08ada2-3e48-4b8b-baba-354f7354fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:59.417937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 688/1390] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:59.715684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3ae01-df03-4238-b484-8b6560b3f406 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:59.716036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3ae01-df03-4238-b484-8b6560b3f406 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:59.716409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a3ae01-df03-4238-b484-8b6560b3f406 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:59.717989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 703/1391] 10.222.0.50 () {54 vars in 962 bytes} [Sat Sep 24 04:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:53:59.724070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd832-10fc-4645-814c-3228f1dfd9d3 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:53:59.724383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd832-10fc-4645-814c-3228f1dfd9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:53:59.724652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fd832-10fc-4645-814c-3228f1dfd9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:53:59.740808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:53:59.803079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3e8fd832-10fc-4645-814c-3228f1dfd9d3 None None] MFA Rules not processed for user `e9754c63d24741be93896d6ef83be3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:53:59.804263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3e8fd832-10fc-4645-814c-3228f1dfd9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:59.878762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 689/1392] 10.222.0.50 () {58 vars in 1059 bytes} [Sat Sep 24 04:53:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:53:59.887767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3ae01-df03-4238-b484-8b6560b3f406 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:53:59.889437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-18a3ae01-df03-4238-b484-8b6560b3f406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:53:59.993371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a3ae01-df03-4238-b484-8b6560b3f406 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:00.052312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:00.068209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:00.068816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:00.069096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:00.069369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:00.070987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:00.182076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:00.183443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55330a-70b2-4646-81c6-2ba2cf2b9478 service glance] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:00.202041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 704/1393] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:00.802196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048bfcd-68b9-41ba-93a1-3735179b12b7 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:00.802781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048bfcd-68b9-41ba-93a1-3735179b12b7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:00.802890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048bfcd-68b9-41ba-93a1-3735179b12b7 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:00.804218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 690/1394] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:00.810325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818dec7-1877-495f-b887-7880c49c2a33 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:00.810871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818dec7-1877-495f-b887-7880c49c2a33 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:00.811020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818dec7-1877-495f-b887-7880c49c2a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:00.882177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c818dec7-1877-495f-b887-7880c49c2a33 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:00.883429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c818dec7-1877-495f-b887-7880c49c2a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:00.926383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:00.952755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aeb96-8605-4991-b047-a47e3ee38c6f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:00.953029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aeb96-8605-4991-b047-a47e3ee38c6f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:00.953922 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30aeb96-8605-4991-b047-a47e3ee38c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:00.987748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 705/1395] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:01.022287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e30aeb96-8605-4991-b047-a47e3ee38c6f None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:01.023506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e30aeb96-8605-4991-b047-a47e3ee38c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:01.077376 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:01.109356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 691/1396] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 04:54:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:54:01.151794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff996c-b9b2-42f3-a263-33eb730f5c46 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:01.152163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff996c-b9b2-42f3-a263-33eb730f5c46 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:01.152509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff996c-b9b2-42f3-a263-33eb730f5c46 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:01.154025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 706/1397] 10.222.0.50 () {54 vars in 943 bytes} [Sat Sep 24 04:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:01.161302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:01.161659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:01.161941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:01.238583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] MFA Rules not processed for user `b8a56d415755492fae33fc13f57c8f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:01.239789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:01.276253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff996c-b9b2-42f3-a263-33eb730f5c46 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:01.277901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0dff996c-b9b2-42f3-a263-33eb730f5c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:01.350019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 692/1398] 10.222.0.50 () {58 vars in 1040 bytes} [Sat Sep 24 04:54:01 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:01.360415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:01.370718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2020072-1958-4f94-be4a-274af5eccf36 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:01.398247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:01.428682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff996c-b9b2-42f3-a263-33eb730f5c46 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:01.454748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:01.476815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:01.477419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:01.477691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:01.477958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:01.479628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:01.544643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:01.545366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:01.545649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:01.545930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:01.548092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:01.563998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:01.565587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e8b8e5-2ffd-42f7-9e50-205ea8392216 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:01.574172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 693/1399] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 04:54:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 04:54:01.765843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:01.767564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4b26c-fb1c-4503-81ff-e45b12eeaada service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:01.787477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 707/1400] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 04:54:04.061926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fe5ea-7ec3-485d-9409-97dbf530ccb8 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:04.063595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fe5ea-7ec3-485d-9409-97dbf530ccb8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:04.064113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fe5ea-7ec3-485d-9409-97dbf530ccb8 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:04.069167 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 694/1401] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:54:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:04.071218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb177722-3b4b-4e9e-9409-7206d30206b1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:04.071995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb177722-3b4b-4e9e-9409-7206d30206b1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:04.072275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb177722-3b4b-4e9e-9409-7206d30206b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:04.113807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:04.155543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-bb177722-3b4b-4e9e-9409-7206d30206b1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:04.157314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-bb177722-3b4b-4e9e-9409-7206d30206b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:04.261823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 708/1402] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:04.552148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fe5ea-7ec3-485d-9409-97dbf530ccb8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:04.554071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-dd0fe5ea-7ec3-485d-9409-97dbf530ccb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:04.622822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:04.723887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fe5ea-7ec3-485d-9409-97dbf530ccb8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:04.860538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:04.861639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:04.862173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:04.862612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:04.864225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:04.884971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:04.965371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:04.967119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c000be8-16f1-47c6-8c3d-03b6ed853583 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:04.990300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 695/1403] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:07.216394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7092963-fadc-4bc7-940c-31492c584f73 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:07.216967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7092963-fadc-4bc7-940c-31492c584f73 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:07.216967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7092963-fadc-4bc7-940c-31492c584f73 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:07.218847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 709/1404] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:07.224175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90734c96-e0d1-4408-8bb2-b2099d2cc622 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:07.224643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90734c96-e0d1-4408-8bb2-b2099d2cc622 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:07.224762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90734c96-e0d1-4408-8bb2-b2099d2cc622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:07.300316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-90734c96-e0d1-4408-8bb2-b2099d2cc622 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:07.301527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-90734c96-e0d1-4408-8bb2-b2099d2cc622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:07.400833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:07.409384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 696/1405] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:07.704072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7092963-fadc-4bc7-940c-31492c584f73 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:07.705602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f7092963-fadc-4bc7-940c-31492c584f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:07.817904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:07.865549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7092963-fadc-4bc7-940c-31492c584f73 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:08.002604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:08.003207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:08.003479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:08.003749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:08.005328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:08.088093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:08.109271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:08.111015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94a26be-fb1e-4eb4-a90b-23e164f5ff10 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:08.131358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 710/1406] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:10.352752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cadaa-1131-418a-8d91-15187bcee4ec None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:10.353319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cadaa-1131-418a-8d91-15187bcee4ec None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:10.353319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cadaa-1131-418a-8d91-15187bcee4ec None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:10.355064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 697/1407] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:10.361073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b948c-281c-446e-9259-08d3c2728d21 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:10.361565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b948c-281c-446e-9259-08d3c2728d21 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:10.361814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b948c-281c-446e-9259-08d3c2728d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:10.442516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ba1b948c-281c-446e-9259-08d3c2728d21 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:10.443731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ba1b948c-281c-446e-9259-08d3c2728d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:10.550866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 711/1408] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:10.840519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cadaa-1131-418a-8d91-15187bcee4ec None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:10.842078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1a9cadaa-1131-418a-8d91-15187bcee4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:11.006955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cadaa-1131-418a-8d91-15187bcee4ec None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:11.082961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:11.134409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:11.135023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:11.135300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:11.135569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:11.137169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:11.241346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:11.242983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619322-2b8c-46fc-8948-e54b68279bd4 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:11.260753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 698/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:20.549255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=79 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:21.258429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=64 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:24.282322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc91f4-a686-4936-ad23-5fa0eb4ac5e2 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:24.282887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc91f4-a686-4936-ad23-5fa0eb4ac5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:24.282887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc91f4-a686-4936-ad23-5fa0eb4ac5e2 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:24.284196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 712/1410] 10.222.0.50 () {54 vars in 918 bytes} [Sat Sep 24 04:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:24.289216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13dd72-e88b-4358-a576-2599b526a7e9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:24.289653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13dd72-e88b-4358-a576-2599b526a7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:24.289756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c13dd72-e88b-4358-a576-2599b526a7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:24.364732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0c13dd72-e88b-4358-a576-2599b526a7e9 None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:24.365942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0c13dd72-e88b-4358-a576-2599b526a7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:24.471824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 699/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Sat Sep 24 04:54:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:24.486312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53a02-741f-4004-8fa5-fecdbb87e568 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:24.486596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53a02-741f-4004-8fa5-fecdbb87e568 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:24.486869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd53a02-741f-4004-8fa5-fecdbb87e568 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:24.488181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 713/1412] 10.222.0.50 () {54 vars in 918 bytes} [Sat Sep 24 04:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:24.492849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda96408-9518-447f-b550-b03a3c04484f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:24.493136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda96408-9518-447f-b550-b03a3c04484f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:24.493417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda96408-9518-447f-b550-b03a3c04484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:24.567672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:24.570654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fda96408-9518-447f-b550-b03a3c04484f None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:24.572447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fda96408-9518-447f-b550-b03a3c04484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:24.687215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 700/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Sat Sep 24 04:54:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:27.230102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8312-d668-46fe-861b-ea7e97a7e5f2 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:27.230662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8312-d668-46fe-861b-ea7e97a7e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:27.230662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f8312-d668-46fe-861b-ea7e97a7e5f2 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:27.232062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:27.237430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c7002-9410-41f8-b2ae-ffd1f3ee7529 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:27.237881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c7002-9410-41f8-b2ae-ffd1f3ee7529 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:27.237988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c7002-9410-41f8-b2ae-ffd1f3ee7529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:27.307632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-813c7002-9410-41f8-b2ae-ffd1f3ee7529 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:27.308829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-813c7002-9410-41f8-b2ae-ffd1f3ee7529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:27.363074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:27.405414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:27.695240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8312-d668-46fe-861b-ea7e97a7e5f2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:27.696784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fc3f8312-d668-46fe-861b-ea7e97a7e5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:27.838932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f8312-d668-46fe-861b-ea7e97a7e5f2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:27.912531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:27.951667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:27.952336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:27.952615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:27.952884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:27.954493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:28.042242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:28.043844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9859c1c-1ed3-4349-801e-8620a34328bf service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:28.062107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 715/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:29.193597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfab0e-73ce-4305-87c3-0ebc7de3388a None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:29.194194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfab0e-73ce-4305-87c3-0ebc7de3388a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:29.194194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfab0e-73ce-4305-87c3-0ebc7de3388a None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:29.195555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:29.202565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae8fbf-93fd-4016-b13d-9134ce8decea None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:29.202998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae8fbf-93fd-4016-b13d-9134ce8decea None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:29.203106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae8fbf-93fd-4016-b13d-9134ce8decea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:29.277137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-94ae8fbf-93fd-4016-b13d-9134ce8decea None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:29.278371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-94ae8fbf-93fd-4016-b13d-9134ce8decea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:29.305105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:29.378609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:29.527870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfab0e-73ce-4305-87c3-0ebc7de3388a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:29.529877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d1cfab0e-73ce-4305-87c3-0ebc7de3388a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:29.673819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfab0e-73ce-4305-87c3-0ebc7de3388a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:29.713207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:29.786264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:29.786794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:29.787072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:29.787333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:29.788899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:29.874610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:29.876175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778b451-934d-4319-aa4b-4368da7eb0af service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:29.894436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 703/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:31.009759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127acaf-20e5-429c-8623-a8d3b0714ce7 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:31.010343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127acaf-20e5-429c-8623-a8d3b0714ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:31.010343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c127acaf-20e5-429c-8623-a8d3b0714ce7 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:31.011707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:31.018577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f8966-2a6f-4e4c-b19c-82d4c7e6941c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:31.019039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f8966-2a6f-4e4c-b19c-82d4c7e6941c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:31.019136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f8966-2a6f-4e4c-b19c-82d4c7e6941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:31.073672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:31.096025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-709f8966-2a6f-4e4c-b19c-82d4c7e6941c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:31.097237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-709f8966-2a6f-4e4c-b19c-82d4c7e6941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:31.191975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:31.344784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127acaf-20e5-429c-8623-a8d3b0714ce7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:31.346249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c127acaf-20e5-429c-8623-a8d3b0714ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:31.488619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c127acaf-20e5-429c-8623-a8d3b0714ce7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:31.500430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:31.601144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:31.601666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:31.601939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:31.602236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:31.603806 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:31.692639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:31.694254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803aa0e4-f3c7-4cfc-9214-7c7baf13df77 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:31.712154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 718/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 04:54:32.820549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913477-b4b3-45b7-a573-b91fa347a2d3 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:32.821082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913477-b4b3-45b7-a573-b91fa347a2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:32.821082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c913477-b4b3-45b7-a573-b91fa347a2d3 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:32.822532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:32.828855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ed4f3-d0ee-4fa4-b8de-8cb8f0d8d90f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:32.829266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ed4f3-d0ee-4fa4-b8de-8cb8f0d8d90f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:32.829400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ed4f3-d0ee-4fa4-b8de-8cb8f0d8d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:32.848548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:32.904830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d43ed4f3-d0ee-4fa4-b8de-8cb8f0d8d90f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:32.905975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d43ed4f3-d0ee-4fa4-b8de-8cb8f0d8d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:33.002071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:33.152420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913477-b4b3-45b7-a573-b91fa347a2d3 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:33.153981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6c913477-b4b3-45b7-a573-b91fa347a2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:33.262057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:33.298093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c913477-b4b3-45b7-a573-b91fa347a2d3 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:33.413024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:33.413624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:33.413896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:33.414198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:33.415791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:33.491959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:33.506156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:33.507753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f9a97-898d-4747-a010-3ca2398d7013 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:33.525536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 706/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:34.631487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf1cf-8af9-47e3-b545-dfd31191f8d9 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:34.632055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf1cf-8af9-47e3-b545-dfd31191f8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:34.632055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacf1cf-8af9-47e3-b545-dfd31191f8d9 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:34.633378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:34.640494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84964d-7106-4519-9d91-ffcf9d6808a2 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:34.640933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84964d-7106-4519-9d91-ffcf9d6808a2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:34.641062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a84964d-7106-4519-9d91-ffcf9d6808a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:34.719338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0a84964d-7106-4519-9d91-ffcf9d6808a2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:34.720541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0a84964d-7106-4519-9d91-ffcf9d6808a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:34.814239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:34.962887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacf1cf-8af9-47e3-b545-dfd31191f8d9 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:34.964436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3eacf1cf-8af9-47e3-b545-dfd31191f8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:35.048168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:35.112404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacf1cf-8af9-47e3-b545-dfd31191f8d9 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:35.223907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:35.224762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:35.225040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:35.225304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:35.226916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:35.275120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:35.315820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:35.317336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b64bf5-0772-46d0-abbb-88f830406cf3 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:35.335540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 721/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:36.386469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa799d-05f9-412c-b9df-b7430749f401 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:36.387002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa799d-05f9-412c-b9df-b7430749f401 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:36.387096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa799d-05f9-412c-b9df-b7430749f401 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:36.388390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:36.395057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e4184-b65d-4e15-975e-eaf5b96549d9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:36.395496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e4184-b65d-4e15-975e-eaf5b96549d9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:36.395922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e4184-b65d-4e15-975e-eaf5b96549d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:36.473114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-675e4184-b65d-4e15-975e-eaf5b96549d9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:36.474360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-675e4184-b65d-4e15-975e-eaf5b96549d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:36.568387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:36.716750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa799d-05f9-412c-b9df-b7430749f401 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:36.718302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-35aa799d-05f9-412c-b9df-b7430749f401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:36.765634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:36.861337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa799d-05f9-412c-b9df-b7430749f401 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:36.970941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:36.971543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:36.971819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:36.972087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:36.973672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:36.980527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:37.064437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:37.066045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d51b03-6081-44e3-ae0c-30d4f3488bed service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:37.084297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 709/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:38.137309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200961ab-6021-4cdf-a4da-a59378ad9261 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:38.137875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200961ab-6021-4cdf-a4da-a59378ad9261 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:38.137875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200961ab-6021-4cdf-a4da-a59378ad9261 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:38.139283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:38.145676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819bdce-fb40-4ee2-8f8e-25d4703c7426 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:38.146108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819bdce-fb40-4ee2-8f8e-25d4703c7426 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:38.146264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819bdce-fb40-4ee2-8f8e-25d4703c7426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:38.220458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2819bdce-fb40-4ee2-8f8e-25d4703c7426 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:38.221665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2819bdce-fb40-4ee2-8f8e-25d4703c7426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:38.289243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:38.319661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:38.471195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200961ab-6021-4cdf-a4da-a59378ad9261 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:38.472639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-200961ab-6021-4cdf-a4da-a59378ad9261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:38.493451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:38.628706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200961ab-6021-4cdf-a4da-a59378ad9261 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:38.715498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:38.741285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:38.741817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:38.742125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:38.742400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:38.743998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:38.831422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:38.833005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328695-9bf7-4bd2-b98b-69e8fe959c3b service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:38.851224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 724/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:39.895491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d51aa-e71a-4eb3-8ed7-1b8366229176 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:39.896044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d51aa-e71a-4eb3-8ed7-1b8366229176 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:39.896044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d51aa-e71a-4eb3-8ed7-1b8366229176 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:39.897357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:39.903497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4f87-a257-408b-99f2-0542c333f45b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:39.903943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4f87-a257-408b-99f2-0542c333f45b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:39.904048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4f87-a257-408b-99f2-0542c333f45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:39.978161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-02af4f87-a257-408b-99f2-0542c333f45b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:39.979366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-02af4f87-a257-408b-99f2-0542c333f45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:40.020378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:40.074731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:40.226280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d51aa-e71a-4eb3-8ed7-1b8366229176 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:40.227804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-df5d51aa-e71a-4eb3-8ed7-1b8366229176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:40.372051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d51aa-e71a-4eb3-8ed7-1b8366229176 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:40.424573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:40.482265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:40.482799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:40.483073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:40.483337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:40.484923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:40.574465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:40.576057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45145fcd-82e6-44b5-9165-df03035d68f5 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:40.593946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 712/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:41.663252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32fd25-b60e-4bab-a928-99c79948dd20 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:41.663652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32fd25-b60e-4bab-a928-99c79948dd20 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:41.663842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32fd25-b60e-4bab-a928-99c79948dd20 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:41.665262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:41.671641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be851d9f-a83a-4188-bb02-25f1eb6d1e91 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:41.672097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be851d9f-a83a-4188-bb02-25f1eb6d1e91 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:41.672205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be851d9f-a83a-4188-bb02-25f1eb6d1e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:41.749056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-be851d9f-a83a-4188-bb02-25f1eb6d1e91 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:41.750299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-be851d9f-a83a-4188-bb02-25f1eb6d1e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:41.752327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:41.844983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:54:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:41.996714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32fd25-b60e-4bab-a928-99c79948dd20 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:41.998317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eb32fd25-b60e-4bab-a928-99c79948dd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:42.141357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32fd25-b60e-4bab-a928-99c79948dd20 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:42.169174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:42.253476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:42.254035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:42.254313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:42.254581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:42.256177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:42.344316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:42.345904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae5e62-2b0d-43d2-b279-0c1b245caf6e service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:42.364761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 727/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:44.423513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eae7b-ae18-4011-89a5-7fbbf5cbd66c None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:44.424178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eae7b-ae18-4011-89a5-7fbbf5cbd66c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:44.424178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eae7b-ae18-4011-89a5-7fbbf5cbd66c None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:44.425429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:44.432303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d196e6-d5d5-4873-b6c4-ffb3ab8767ea None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:44.432736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d196e6-d5d5-4873-b6c4-ffb3ab8767ea None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:44.433057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d196e6-d5d5-4873-b6c4-ffb3ab8767ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:44.479799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:44.505769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-44d196e6-d5d5-4873-b6c4-ffb3ab8767ea None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:44.507020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-44d196e6-d5d5-4873-b6c4-ffb3ab8767ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:44.602903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:44.896470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eae7b-ae18-4011-89a5-7fbbf5cbd66c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:44.898033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-db9eae7b-ae18-4011-89a5-7fbbf5cbd66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:44.999466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9eae7b-ae18-4011-89a5-7fbbf5cbd66c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:45.031958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:45.075416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:45.076031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:45.076306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:45.076575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:45.078215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:45.197372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:45.199467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e5c00-cb53-4b2f-8085-67545daa100f service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:45.220020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 715/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:47.819590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b2bbb-45ce-4a6f-8d47-bf6ebef371a3 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:47.820141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b2bbb-45ce-4a6f-8d47-bf6ebef371a3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:47.820141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b2bbb-45ce-4a6f-8d47-bf6ebef371a3 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:47.821482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:47.826864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7caa580-1cb8-4d24-bd79-fd6b1063ddf5 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:47.827261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7caa580-1cb8-4d24-bd79-fd6b1063ddf5 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:47.827407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7caa580-1cb8-4d24-bd79-fd6b1063ddf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:47.895120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a7caa580-1cb8-4d24-bd79-fd6b1063ddf5 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:47.896328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a7caa580-1cb8-4d24-bd79-fd6b1063ddf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:47.910800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:47.999252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:48.287505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b2bbb-45ce-4a6f-8d47-bf6ebef371a3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:48.289036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3c4b2bbb-45ce-4a6f-8d47-bf6ebef371a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:48.391254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b2bbb-45ce-4a6f-8d47-bf6ebef371a3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:48.403844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:48.466065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:48.466672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:48.466948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:48.467217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:48.468818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:48.576259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:48.577617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e31c5-31ec-49a5-90f2-1ce50313b57d service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:48.595992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 730/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:51.059154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f53b5-00aa-4879-97ab-6a0bdc0a5c69 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:51.059722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f53b5-00aa-4879-97ab-6a0bdc0a5c69 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:51.059722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f53b5-00aa-4879-97ab-6a0bdc0a5c69 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:51.061105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:51.066365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d5ed-5971-45ad-9941-bf1035d02dbd None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:51.066796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d5ed-5971-45ad-9941-bf1035d02dbd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:51.066918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d5ed-5971-45ad-9941-bf1035d02dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:51.114337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:51.132671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c214d5ed-5971-45ad-9941-bf1035d02dbd None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:51.133882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c214d5ed-5971-45ad-9941-bf1035d02dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:51.228737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:51.513096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f53b5-00aa-4879-97ab-6a0bdc0a5c69 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:51.514698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-597f53b5-00aa-4879-97ab-6a0bdc0a5c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:51.617801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f53b5-00aa-4879-97ab-6a0bdc0a5c69 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:51.658891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:51.693946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:51.694571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:51.694847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:51.695116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:51.696701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:51.810617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:51.811958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f7391-6b5d-4d5f-987c-e12fc82e3bef service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:51.829471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 718/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:54.069039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab38a09-8bc5-493f-b79a-b9c0b614da2c None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:54.069576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab38a09-8bc5-493f-b79a-b9c0b614da2c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:54.069576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab38a09-8bc5-493f-b79a-b9c0b614da2c None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:54.070975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:54.076776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a617-1d9a-43fa-8354-21053f1f99a0 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:54.077179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a617-1d9a-43fa-8354-21053f1f99a0 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:54.077315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a617-1d9a-43fa-8354-21053f1f99a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:54.142526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f102a617-1d9a-43fa-8354-21053f1f99a0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:54.143716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f102a617-1d9a-43fa-8354-21053f1f99a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:54.163405 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:54.240085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:54.525862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab38a09-8bc5-493f-b79a-b9c0b614da2c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:54.527430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eab38a09-8bc5-493f-b79a-b9c0b614da2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:54.631306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab38a09-8bc5-493f-b79a-b9c0b614da2c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:54.652862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:54.708635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:54.709242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:54.709522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:54.709794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:54.711424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:54.821655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:54.823050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceefce0-1768-496e-b638-611dd43bf2e5 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:54.841531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 733/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:54:57.057320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9124dc-6a09-4bdc-bcdd-ae085f7c05a4 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:57.057930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9124dc-6a09-4bdc-bcdd-ae085f7c05a4 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:57.058139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9124dc-6a09-4bdc-bcdd-ae085f7c05a4 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:57.059534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:54:57.064773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabc83c-94fa-4405-9f70-aeddebc63c2f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:57.065226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabc83c-94fa-4405-9f70-aeddebc63c2f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:57.065330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabc83c-94fa-4405-9f70-aeddebc63c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:57.137361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:57.140879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5fabc83c-94fa-4405-9f70-aeddebc63c2f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:54:57.142934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5fabc83c-94fa-4405-9f70-aeddebc63c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:57.238593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:54:57.524324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9124dc-6a09-4bdc-bcdd-ae085f7c05a4 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:54:57.526452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8f9124dc-6a09-4bdc-bcdd-ae085f7c05a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:57.628101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9124dc-6a09-4bdc-bcdd-ae085f7c05a4 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:54:57.679138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:54:57.704079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:54:57.704683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:54:57.704961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:54:57.705235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:54:57.706874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:54:57.815004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:54:57.816370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a2581-6b09-4695-9e78-4a41146a7f35 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:54:57.834983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 721/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:00.052253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0578b-1a2c-4f68-a646-4ea7fda73036 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:00.052846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0578b-1a2c-4f68-a646-4ea7fda73036 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:00.052846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0578b-1a2c-4f68-a646-4ea7fda73036 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:00.054222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:00.060178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522eca-8020-44a9-ba0b-80658743170d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:00.060619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522eca-8020-44a9-ba0b-80658743170d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:00.060717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1522eca-8020-44a9-ba0b-80658743170d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:00.133322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d1522eca-8020-44a9-ba0b-80658743170d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:00.134641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d1522eca-8020-44a9-ba0b-80658743170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:00.167943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:00.234596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:00.524148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0578b-1a2c-4f68-a646-4ea7fda73036 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:00.526303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-bcc0578b-1a2c-4f68-a646-4ea7fda73036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:00.629210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc0578b-1a2c-4f68-a646-4ea7fda73036 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:00.662439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:00.707844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:00.708443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:00.708715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:00.708981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:00.710582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:00.819184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:00.820525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03594-fbf1-4e0a-a9e3-4867ccf2dfd2 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:00.839551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 736/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:03.123902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc9e-ff0b-464c-b800-9fbfee1a380d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:03.124445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc9e-ff0b-464c-b800-9fbfee1a380d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:03.124445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cbc9e-ff0b-464c-b800-9fbfee1a380d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:03.125801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:03.131044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f5c6-2c8c-4a03-b972-d6d544d686ab None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:03.131447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f5c6-2c8c-4a03-b972-d6d544d686ab None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:03.131596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f5c6-2c8c-4a03-b972-d6d544d686ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:03.196559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9d77f5c6-2c8c-4a03-b972-d6d544d686ab None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:03.198393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9d77f5c6-2c8c-4a03-b972-d6d544d686ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:03.206485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:03.293532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:03.574850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc9e-ff0b-464c-b800-9fbfee1a380d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:03.576384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-252cbc9e-ff0b-464c-b800-9fbfee1a380d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:03.685697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cbc9e-ff0b-464c-b800-9fbfee1a380d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:03.741593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:03.765434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:03.766402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:03.766706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:03.766981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:03.768589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:03.881717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:03.883096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b7ea9-8568-4663-b30e-f07093ed0f8c service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:03.901368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 724/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:06.083760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4326446-6caa-4418-ba09-814b3dcd1eeb None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:06.084326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4326446-6caa-4418-ba09-814b3dcd1eeb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:06.084326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4326446-6caa-4418-ba09-814b3dcd1eeb None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:06.085661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:06.091628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b820d-ffb7-485d-b00e-5d67fee81b46 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:06.092029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b820d-ffb7-485d-b00e-5d67fee81b46 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:06.092178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b820d-ffb7-485d-b00e-5d67fee81b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:06.273836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-652b820d-ffb7-485d-b00e-5d67fee81b46 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:06.275090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-652b820d-ffb7-485d-b00e-5d67fee81b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:06.313268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:06.373013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:06.658680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4326446-6caa-4418-ba09-814b3dcd1eeb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:06.660221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a4326446-6caa-4418-ba09-814b3dcd1eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:06.762254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4326446-6caa-4418-ba09-814b3dcd1eeb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:06.805572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:06.842965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:06.843576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:06.843856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:06.844129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:06.845724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:06.953310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:06.954703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1744ae-dc9e-48b1-995b-8fd9fc421ae5 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:06.972713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 739/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:09.128507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf4178-2b38-412c-8928-17e083f16421 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:09.129071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf4178-2b38-412c-8928-17e083f16421 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:09.129071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cf4178-2b38-412c-8928-17e083f16421 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:09.130423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:09.135739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5808d-0ccd-4dfc-a881-80322c4b8644 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:09.136200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5808d-0ccd-4dfc-a881-80322c4b8644 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:09.136317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5808d-0ccd-4dfc-a881-80322c4b8644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:09.200441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-26e5808d-0ccd-4dfc-a881-80322c4b8644 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:09.201630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-26e5808d-0ccd-4dfc-a881-80322c4b8644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:09.219447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:09.297430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:09.576848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf4178-2b38-412c-8928-17e083f16421 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:09.578428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-34cf4178-2b38-412c-8928-17e083f16421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:09.684186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cf4178-2b38-412c-8928-17e083f16421 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:09.750132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:09.756672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:09.757273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:09.757551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:09.757819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:09.759459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:09.866419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:09.867767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee246927-a91f-4aee-8325-6d8dd05543a9 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:09.885441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 727/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:12.053560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f109b0-af79-496c-b2b9-2d0aaa2ac0ba None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:12.054157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f109b0-af79-496c-b2b9-2d0aaa2ac0ba None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:12.054157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f109b0-af79-496c-b2b9-2d0aaa2ac0ba None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:12.055607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:12.060966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b990a-2043-46e1-8b1a-066a7e7a2e55 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:12.061357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b990a-2043-46e1-8b1a-066a7e7a2e55 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:12.061526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b990a-2043-46e1-8b1a-066a7e7a2e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:12.126135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-006b990a-2043-46e1-8b1a-066a7e7a2e55 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:12.127329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-006b990a-2043-46e1-8b1a-066a7e7a2e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:12.175996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:12.224222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:12.504928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f109b0-af79-496c-b2b9-2d0aaa2ac0ba None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:12.506537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-07f109b0-af79-496c-b2b9-2d0aaa2ac0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:12.609631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f109b0-af79-496c-b2b9-2d0aaa2ac0ba None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:12.657986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:12.684655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:12.685258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:12.685535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:12.685803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:12.687423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:12.792662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:12.794033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575dbcc-1f97-4fb5-8913-d78022cc243d service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:12.811851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 742/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:22.222693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=22 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:22.809639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=66 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:32.702061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973cd8-be58-444c-9084-71e4a5138e57 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:32.702579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973cd8-be58-444c-9084-71e4a5138e57 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:32.702697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973cd8-be58-444c-9084-71e4a5138e57 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:32.703939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 729/1471] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:32.709176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6329e02-a75f-443d-a5e9-092ee0a06535 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:32.709601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6329e02-a75f-443d-a5e9-092ee0a06535 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:32.709776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6329e02-a75f-443d-a5e9-092ee0a06535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:32.774954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c6329e02-a75f-443d-a5e9-092ee0a06535 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:32.776148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c6329e02-a75f-443d-a5e9-092ee0a06535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:32.869745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 743/1472] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:55:33.168740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6696b3-0fe6-47b6-adf0-3ea34b1798e9 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:33.169016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6696b3-0fe6-47b6-adf0-3ea34b1798e9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:33.169282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6696b3-0fe6-47b6-adf0-3ea34b1798e9 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:33.170632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 730/1473] 10.222.0.50 () {54 vars in 962 bytes} [Sat Sep 24 04:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:33.175489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f7c82-9219-428b-a08b-472227743c11 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:33.175802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f7c82-9219-428b-a08b-472227743c11 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:33.176080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f7c82-9219-428b-a08b-472227743c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:33.250598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-876f7c82-9219-428b-a08b-472227743c11 None None] MFA Rules not processed for user `e9754c63d24741be93896d6ef83be3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:33.251801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-876f7c82-9219-428b-a08b-472227743c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:33.276068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:33.331060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 744/1474] 10.222.0.50 () {58 vars in 1059 bytes} [Sat Sep 24 04:55:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:33.339324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6696b3-0fe6-47b6-adf0-3ea34b1798e9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:33.340838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-df6696b3-0fe6-47b6-adf0-3ea34b1798e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:33.442335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6696b3-0fe6-47b6-adf0-3ea34b1798e9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:33.514189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:33.515042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:33.515312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:33.515573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:33.517147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:33.575342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:33.629836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:33.631287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436cb411-de40-487f-91ef-baaf83383ae8 service glance] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:33.651899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 731/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:35.866360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c4c0e-ded3-4884-af0e-0eba3bb02b92 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:35.866920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c4c0e-ded3-4884-af0e-0eba3bb02b92 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:35.866920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c4c0e-ded3-4884-af0e-0eba3bb02b92 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:35.868291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 745/1476] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:35.873687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b0d71-dd27-48d4-a199-1075dec4f5ff None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:35.874128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b0d71-dd27-48d4-a199-1075dec4f5ff None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:35.874274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b0d71-dd27-48d4-a199-1075dec4f5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:35.939227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e93b0d71-dd27-48d4-a199-1075dec4f5ff None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:35.940338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e93b0d71-dd27-48d4-a199-1075dec4f5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:36.039512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 732/1477] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:36.325662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c4c0e-ded3-4884-af0e-0eba3bb02b92 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:36.327242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5e7c4c0e-ded3-4884-af0e-0eba3bb02b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:36.455880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c4c0e-ded3-4884-af0e-0eba3bb02b92 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:36.504862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:36.548344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:36.548954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:36.549230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:36.549502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:36.550164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:36.551895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:55:36.551895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 04:55:36.551895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 04:55:36.553036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebd966-e11b-473e-808f-eead043e447a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:36.572286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 746/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Sat Sep 24 04:55:36 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 04:55:38.755541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142e94-2e14-4c63-9c34-0881eb888f70 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:38.756066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142e94-2e14-4c63-9c34-0881eb888f70 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:38.756066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d142e94-2e14-4c63-9c34-0881eb888f70 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:38.757451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 733/1479] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:38.762874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879fc8-1bee-4455-9014-21dc96b3d342 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:38.763291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879fc8-1bee-4455-9014-21dc96b3d342 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:38.763429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8879fc8-1bee-4455-9014-21dc96b3d342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:38.830244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e8879fc8-1bee-4455-9014-21dc96b3d342 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:38.831460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e8879fc8-1bee-4455-9014-21dc96b3d342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:38.933776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 747/1480] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:39.228134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142e94-2e14-4c63-9c34-0881eb888f70 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:39.229661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6d142e94-2e14-4c63-9c34-0881eb888f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:39.239264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:39.373018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d142e94-2e14-4c63-9c34-0881eb888f70 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:39.451415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:39.483659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:39.484196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:39.484470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:39.484739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:39.486358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:39.574212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:39.575772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aff17f3-25ca-4aa0-b57e-8df676a719ff service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:39.592826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 734/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:41.912917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8af9-f11d-4b32-a10d-9a359ac6efb6 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:41.913439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8af9-f11d-4b32-a10d-9a359ac6efb6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:41.913439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e8af9-f11d-4b32-a10d-9a359ac6efb6 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:41.914778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 748/1482] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:41.919934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ff39-1dfb-4acc-90cc-9978a563a58a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:41.920381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ff39-1dfb-4acc-90cc-9978a563a58a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:41.920528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ff39-1dfb-4acc-90cc-9978a563a58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:41.984976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4a17ff39-1dfb-4acc-90cc-9978a563a58a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:41.986198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4a17ff39-1dfb-4acc-90cc-9978a563a58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:42.032041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:42.083519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 735/1483] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:42.367155 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e8af9-f11d-4b32-a10d-9a359ac6efb6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:42.368946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e72e8af9-f11d-4b32-a10d-9a359ac6efb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:42.392421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:42.471619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e8af9-f11d-4b32-a10d-9a359ac6efb6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:42.543508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:42.544135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:42.544410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:42.544674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:42.546259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:42.617466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:42.664574 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:42.666032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1b523-b68b-45c0-b045-c4b61a048ab5 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:42.683708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 749/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:44.862644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb388-5dc3-4d5d-b7b8-d02dd2359e20 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:44.863199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb388-5dc3-4d5d-b7b8-d02dd2359e20 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:44.863199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb388-5dc3-4d5d-b7b8-d02dd2359e20 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:44.864517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 736/1485] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:44.869904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8beee-6b7a-499c-9261-8df2d1e63ef8 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:44.870352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8beee-6b7a-499c-9261-8df2d1e63ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:44.870504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf8beee-6b7a-499c-9261-8df2d1e63ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:44.937141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9bf8beee-6b7a-499c-9261-8df2d1e63ef8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:44.938342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9bf8beee-6b7a-499c-9261-8df2d1e63ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:45.034137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 750/1486] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:45.330232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb388-5dc3-4d5d-b7b8-d02dd2359e20 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:45.331793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6effb388-5dc3-4d5d-b7b8-d02dd2359e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:45.433641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb388-5dc3-4d5d-b7b8-d02dd2359e20 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:45.503843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:45.505644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:45.506514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:45.506893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:45.507177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:45.509164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:45.624973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:45.626425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434d805-7794-4291-9c81-f0b43fdb2e84 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:45.658196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 737/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:47.983197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e8446-69f3-43fb-86dd-6d5a5bd0f0e5 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:47.983635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e8446-69f3-43fb-86dd-6d5a5bd0f0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:47.983875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e8446-69f3-43fb-86dd-6d5a5bd0f0e5 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:47.985369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 751/1488] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 04:55:47.990859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119a75e-feac-466e-9c1c-4a168368ff11 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:47.991379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119a75e-feac-466e-9c1c-4a168368ff11 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:47.991484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119a75e-feac-466e-9c1c-4a168368ff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:48.055824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d119a75e-feac-466e-9c1c-4a168368ff11 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:48.057019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d119a75e-feac-466e-9c1c-4a168368ff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:48.107842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:48.152377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 738/1489] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:48.446048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e8446-69f3-43fb-86dd-6d5a5bd0f0e5 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:48.447598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e58e8446-69f3-43fb-86dd-6d5a5bd0f0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:48.483373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:48.550919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e8446-69f3-43fb-86dd-6d5a5bd0f0e5 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:48.627132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:48.627745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:48.628024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:48.628296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:48.629887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:48.709507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:48.739060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:48.741159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9ff0c-b7eb-4d1b-8b74-6f53f481aadb service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:48.758856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 752/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:55:51.069959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2737ef-5c45-4d41-a763-3358a3606847 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:51.070542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2737ef-5c45-4d41-a763-3358a3606847 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:51.070542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2737ef-5c45-4d41-a763-3358a3606847 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:51.071871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 739/1491] 10.222.0.50 () {54 vars in 927 bytes} [Sat Sep 24 04:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:55:51.077327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32e454-55dc-47ec-bf00-669094398611 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:51.077709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32e454-55dc-47ec-bf00-669094398611 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:51.077911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d32e454-55dc-47ec-bf00-669094398611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:51.148190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1d32e454-55dc-47ec-bf00-669094398611 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:55:51.149455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1d32e454-55dc-47ec-bf00-669094398611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:51.246679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 753/1492] 10.222.0.50 () {58 vars in 1024 bytes} [Sat Sep 24 04:55:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:55:51.553580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2737ef-5c45-4d41-a763-3358a3606847 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:55:51.555139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7f2737ef-5c45-4d41-a763-3358a3606847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:51.667417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2737ef-5c45-4d41-a763-3358a3606847 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:55:51.739752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:55:51.740987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:55:51.741443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:55:51.741881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:55:51.743698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:55:51.752234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:55:51.853926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:55:51.855468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225cefe-9a89-4d5a-9598-baaab7d489b5 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:55:51.873230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 740/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:56:01.245108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=82 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:56:01.871004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=50 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:11.149458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:11.150076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:11.150076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:11.223952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:57:11.225148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:11.318561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 754/1494] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 04:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:57:11.326515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:11.327004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:11.327124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:11.402747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:57:11.403881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:11.498405 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 741/1495] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 04:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:57:11.503124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:11.504321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:11.517213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c99c34-fe23-4a23-821d-5a2f6fd4e9c7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:11.577810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:11.611567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:11.612168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:11.612445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:11.612714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:11.613348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 04:57:11.613644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:11.614687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:11.614687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 04:57:11.614687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 04:57:11.615562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:11.626890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 755/1496] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 04:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:57:11.635256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:11.645022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58206b8f-b56f-4477-8592-c10cc9534773 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:11.706243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:11.740356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:11.740955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:11.741222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:11.741484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:11.742130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:11.743899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:11.743899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 04:57:11.743899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 04:57:11.745062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:11.799776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 742/1497] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 04:57:11 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:57:11.804133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:11.805676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:11.839206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647164f0-1ac9-4102-ae4b-fc6d722fe5aa admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:11.930227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:11.930825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:11.931100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:11.931398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:11.932030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:11.933029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:11.933029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 04:57:11.933029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 04:57:11.934250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:11.975975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 756/1498] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 04:57:11 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 04:57:11.980009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:11.981549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:12.042750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d245c-4975-4671-9143-45fc7c1d2ad9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:12.129857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:12.141496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:12.142132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.142411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.142676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.143309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:12.144305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:12.144305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 04:57:12.144305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 04:57:12.145175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:12.157161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 743/1499] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 04:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:57:12.161362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:12.162869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:12.174490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f9ee0-4fc1-4484-b3a1-0df9dfdc6a1d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:12.199340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:12.269074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:12.269600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.269903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.270207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.270886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:12.271885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:12.271885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 04:57:12.271885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 04:57:12.272759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65bc72-7cbf-46bd-97cf-b94ca2c03a4a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:12.284664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 757/1500] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 04:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 04:57:12.288756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:12.290316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:12.324251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dc59f-7fca-4061-a118-e5c20f0ea4fa admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:12.422980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:12.423675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.423888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.424152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] PATH_INFO: `/v3/projects/9c9205bcdf14477e93bf11232e5ae235/users/eeade7888cb94bac8eb4febbd54c7b9a/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.446501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9205bcdf14477e93bf11232e5ae235, user_id=eeade7888cb94bac8eb4febbd54c7b9a, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:12.447524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:12.447524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 04:57:12.447524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 04:57:12.449422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe77dba-5723-4d17-a017-85c21358b7ec admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:12.473654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 744/1501] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 04:57:12 2022] PUT /identity/v3/projects/9c9205bcdf14477e93bf11232e5ae235/users/eeade7888cb94bac8eb4febbd54c7b9a/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:57:12.486480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c49487-3a6e-4b7f-8cc9-4a566ce6d910 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.486767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c49487-3a6e-4b7f-8cc9-4a566ce6d910 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.487046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c49487-3a6e-4b7f-8cc9-4a566ce6d910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.544117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-86c49487-3a6e-4b7f-8cc9-4a566ce6d910 None None] MFA Rules not processed for user `eeade7888cb94bac8eb4febbd54c7b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:57:12.545259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-86c49487-3a6e-4b7f-8cc9-4a566ce6d910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:12.620982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 758/1502] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 04:57:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:57:12.634447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11664c-b35d-44ce-b079-978e3498072f None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.634738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11664c-b35d-44ce-b079-978e3498072f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.635011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11664c-b35d-44ce-b079-978e3498072f None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.636388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 745/1503] 10.222.0.50 () {54 vars in 963 bytes} [Sat Sep 24 04:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:57:12.641494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead7147-97b3-4235-8692-c9e9382dd4cc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.641781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead7147-97b3-4235-8692-c9e9382dd4cc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.642092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead7147-97b3-4235-8692-c9e9382dd4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.674330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:12.721447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-eead7147-97b3-4235-8692-c9e9382dd4cc None None] MFA Rules not processed for user `3a5e44f909a94c74990b2e80f1da5169`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:57:12.722683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eead7147-97b3-4235-8692-c9e9382dd4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:12.796294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 759/1504] 10.222.0.50 () {58 vars in 1060 bytes} [Sat Sep 24 04:57:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:57:12.804602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11664c-b35d-44ce-b079-978e3498072f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:12.806151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7f11664c-b35d-44ce-b079-978e3498072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:12.855173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:12.909572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11664c-b35d-44ce-b079-978e3498072f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:12.980520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:12.981116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:12.981387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:12.981651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:12.983246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:13.079642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:13.080978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:13.084378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:13.099761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 746/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:57:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:57:13.144773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b9e01-6556-491e-9735-a0abea88725d None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:13.145057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b9e01-6556-491e-9735-a0abea88725d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:13.145330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b9e01-6556-491e-9735-a0abea88725d None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:13.146609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 760/1506] 10.222.0.50 () {52 vars in 830 bytes} [Sat Sep 24 04:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:57:13.150161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:13.151355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:13.247812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeddb5c-6c9b-4d3f-bced-2df4d757a33c service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:13.339739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:13.340295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:13.340572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:13.340842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] PATH_INFO: `/v3/users/eeade7888cb94bac8eb4febbd54c7b9a` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:13.353357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeade7888cb94bac8eb4febbd54c7b9a)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:13.354406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:13.354406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 04:57:13.354406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 04:57:13.355612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:13.365058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:13.421598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeade7888cb94bac8eb4febbd54c7b9a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:13.430745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeade7888cb94bac8eb4febbd54c7b9a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:13.431405 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeade7888cb94bac8eb4febbd54c7b9a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:13.431692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeade7888cb94bac8eb4febbd54c7b9a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:13.445868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeade7888cb94bac8eb4febbd54c7b9a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:13.473191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 747/1507] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 04:57:13 2022] DELETE /identity/v3/users/eeade7888cb94bac8eb4febbd54c7b9a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:57:13.477619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:13.478107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:13.478216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:13.537030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:57:13.538299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:13.617865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:13.631481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 761/1508] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 04:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:57:13.642171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:13.643447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:13.743649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0378a03-4b33-4014-a35f-605c4a70514a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:13.796736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:13.818293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:13.818894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:13.819171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:13.819440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:13.821022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:13.927598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:13.929023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768428d-d638-468e-a08e-a06cb41d5707 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:13.947650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 748/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:57:14.322143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:57:14.323476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:57:14.335948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffafe4-11fe-49b1-9caf-773d6730c045 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:57:14.447377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:57:14.447918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:57:14.448194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:57:14.448463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] PATH_INFO: `/v3/projects/9c9205bcdf14477e93bf11232e5ae235` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:57:14.459066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9205bcdf14477e93bf11232e5ae235)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:57:14.460098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 04:57:14.460098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 04:57:14.460098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 04:57:14.461294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:57:14.517893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9205bcdf14477e93bf11232e5ae235', 'request_id': 'req-e0c72810-05cf-44ba-a0f0-adb5616140a7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:14.526714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9205bcdf14477e93bf11232e5ae235', 'request_id': 'req-e0c72810-05cf-44ba-a0f0-adb5616140a7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:14.532700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9205bcdf14477e93bf11232e5ae235', 'request_id': 'req-e0c72810-05cf-44ba-a0f0-adb5616140a7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:14.561702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] The token cache is being invalidate because project 9c9205bcdf14477e93bf11232e5ae235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 04:57:14.561987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:14.562523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e0c72810-05cf-44ba-a0f0-adb5616140a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 04:57:14.564512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 762/1510] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 04:57:14 2022] DELETE /identity/v3/projects/9c9205bcdf14477e93bf11232e5ae235 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 04:57:23.945491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=62 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:57:24.559794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:58:41.546925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eb986-6c8d-4b55-b9f1-813ae78d249f None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:41.547523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eb986-6c8d-4b55-b9f1-813ae78d249f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:41.547523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9eb986-6c8d-4b55-b9f1-813ae78d249f None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:41.548792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:58:41.553820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909887d-2345-491e-8d41-a8fbf1d952fe None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:41.554261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909887d-2345-491e-8d41-a8fbf1d952fe None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:41.554405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909887d-2345-491e-8d41-a8fbf1d952fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:41.639698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f909887d-2345-491e-8d41-a8fbf1d952fe None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:58:41.640932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f909887d-2345-491e-8d41-a8fbf1d952fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:41.747241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:58:41.901761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9eb986-6c8d-4b55-b9f1-813ae78d249f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:58:41.903994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ba9eb986-6c8d-4b55-b9f1-813ae78d249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:42.029141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9eb986-6c8d-4b55-b9f1-813ae78d249f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:58:42.035073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c083d-317a-44cd-b70f-8d3d4f97350e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:58:42.035672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c083d-317a-44cd-b70f-8d3d4f97350e service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:42.035943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c083d-317a-44cd-b70f-8d3d4f97350e service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:42.036208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c083d-317a-44cd-b70f-8d3d4f97350e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:42.044067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c083d-317a-44cd-b70f-8d3d4f97350e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:58:42.045684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c083d-317a-44cd-b70f-8d3d4f97350e service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:58:42.055568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 750/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:58:43.135499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545cb50-8e50-4113-9032-1b159967806f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:43.136047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545cb50-8e50-4113-9032-1b159967806f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:43.136047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545cb50-8e50-4113-9032-1b159967806f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:43.137627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:58:43.143601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8943a-2762-4314-8f95-551ee93387eb None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:43.144013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8943a-2762-4314-8f95-551ee93387eb None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:43.144117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8943a-2762-4314-8f95-551ee93387eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:43.163617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c8c8943a-2762-4314-8f95-551ee93387eb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:58:43.164801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c8c8943a-2762-4314-8f95-551ee93387eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:43.173797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:58:43.322691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545cb50-8e50-4113-9032-1b159967806f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:58:43.336738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545cb50-8e50-4113-9032-1b159967806f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:58:43.342069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c362554-a970-4637-bb67-456d15821025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:58:43.342658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362554-a970-4637-bb67-456d15821025 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:43.343457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362554-a970-4637-bb67-456d15821025 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:43.343733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c362554-a970-4637-bb67-456d15821025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:43.351545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362554-a970-4637-bb67-456d15821025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:58:43.353108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c362554-a970-4637-bb67-456d15821025 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:58:43.362771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 765/1516] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:58:43.562350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273646c4-87ea-49f3-b7b7-1f69de15de47 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:43.562630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273646c4-87ea-49f3-b7b7-1f69de15de47 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:43.562899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273646c4-87ea-49f3-b7b7-1f69de15de47 None None] PATH_INFO: `/v3` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:43.564328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 752/1517] 10.222.0.16 () {54 vars in 925 bytes} [Sat Sep 24 04:58:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 04:58:43.570568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766f0a-fa52-416f-8af9-e88e1fb7767b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:43.570856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766f0a-fa52-416f-8af9-e88e1fb7767b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:43.571136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4766f0a-fa52-416f-8af9-e88e1fb7767b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:43.631074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f4766f0a-fa52-416f-8af9-e88e1fb7767b None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:58:43.632262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f4766f0a-fa52-416f-8af9-e88e1fb7767b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:43.649366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:58:43.704427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 766/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Sep 24 04:58:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:58:47.352324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886a13c-d4bf-487a-9235-932cbffb66da None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:47.352874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886a13c-d4bf-487a-9235-932cbffb66da None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:47.352874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886a13c-d4bf-487a-9235-932cbffb66da None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:47.354278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 753/1519] 10.222.0.50 () {54 vars in 961 bytes} [Sat Sep 24 04:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:58:47.359338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384022d-2616-414f-9846-3128f6997403 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:47.359738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384022d-2616-414f-9846-3128f6997403 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:47.359872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384022d-2616-414f-9846-3128f6997403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:47.379356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6384022d-2616-414f-9846-3128f6997403 None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:58:47.380617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6384022d-2616-414f-9846-3128f6997403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:47.390665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 767/1520] 10.222.0.50 () {58 vars in 1058 bytes} [Sat Sep 24 04:58:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:58:47.398245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886a13c-d4bf-487a-9235-932cbffb66da None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:58:47.412385 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0886a13c-d4bf-487a-9235-932cbffb66da None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:58:47.417345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b229be2-62c7-4faa-a304-817b4688694e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:58:47.417938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b229be2-62c7-4faa-a304-817b4688694e service placement] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:47.418251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b229be2-62c7-4faa-a304-817b4688694e service placement] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:47.418517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b229be2-62c7-4faa-a304-817b4688694e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:47.426844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b229be2-62c7-4faa-a304-817b4688694e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:58:47.428412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b229be2-62c7-4faa-a304-817b4688694e service placement] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:58:47.437939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 754/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:58:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:58:54.826622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441686cf-4c16-4149-b126-d169d5c949da None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:54.827729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441686cf-4c16-4149-b126-d169d5c949da None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:54.827729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441686cf-4c16-4149-b126-d169d5c949da None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:54.829037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:58:54.835229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9a5e-5ff9-4457-8911-b88ae7321a34 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:54.835640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9a5e-5ff9-4457-8911-b88ae7321a34 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:54.835794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c9a5e-5ff9-4457-8911-b88ae7321a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:54.855249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-cf4c9a5e-5ff9-4457-8911-b88ae7321a34 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:58:54.856421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cf4c9a5e-5ff9-4457-8911-b88ae7321a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:54.864075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:58:55.017451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441686cf-4c16-4149-b126-d169d5c949da None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:58:55.032232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441686cf-4c16-4149-b126-d169d5c949da None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:58:55.037730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515bf4-2d91-4336-ba61-c42d23e66d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:58:55.038364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515bf4-2d91-4336-ba61-c42d23e66d78 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:55.038665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515bf4-2d91-4336-ba61-c42d23e66d78 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:55.038962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515bf4-2d91-4336-ba61-c42d23e66d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:55.047012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515bf4-2d91-4336-ba61-c42d23e66d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:58:55.048578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515bf4-2d91-4336-ba61-c42d23e66d78 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:58:55.057834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 769/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:58:56.133527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6118eca-5bdf-4559-a473-8b52cb9ee204 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:56.134120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6118eca-5bdf-4559-a473-8b52cb9ee204 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:56.134120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6118eca-5bdf-4559-a473-8b52cb9ee204 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:56.135437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 24 04:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:58:56.142346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947fbb5-fd33-4841-908a-9664cb9cf09c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:56.142778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947fbb5-fd33-4841-908a-9664cb9cf09c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:56.142897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947fbb5-fd33-4841-908a-9664cb9cf09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:56.162225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9947fbb5-fd33-4841-908a-9664cb9cf09c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:58:56.163421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9947fbb5-fd33-4841-908a-9664cb9cf09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:58:56.171194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 24 04:58:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:58:56.322240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6118eca-5bdf-4559-a473-8b52cb9ee204 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:58:56.336962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6118eca-5bdf-4559-a473-8b52cb9ee204 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:58:56.342448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622fde8-6921-484a-87ce-c019419aa42b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:58:56.343591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622fde8-6921-484a-87ce-c019419aa42b service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:58:56.343868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622fde8-6921-484a-87ce-c019419aa42b service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:58:56.344139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622fde8-6921-484a-87ce-c019419aa42b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:58:56.352266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622fde8-6921-484a-87ce-c019419aa42b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:58:56.353849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622fde8-6921-484a-87ce-c019419aa42b service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:58:56.363267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 757/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:59:00.356295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82783479-cee3-431e-a060-386bb21bcca0 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:00.356862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82783479-cee3-431e-a060-386bb21bcca0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:00.356862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82783479-cee3-431e-a060-386bb21bcca0 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:00.358720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:59:00.363987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecb71-d26f-45b2-97be-55852f42a19d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:00.364391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecb71-d26f-45b2-97be-55852f42a19d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:00.364529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecb71-d26f-45b2-97be-55852f42a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:00.384143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-952ecb71-d26f-45b2-97be-55852f42a19d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:59:00.385330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-952ecb71-d26f-45b2-97be-55852f42a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:00.393436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:59:00.678575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82783479-cee3-431e-a060-386bb21bcca0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:59:00.680337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-82783479-cee3-431e-a060-386bb21bcca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:00.758580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82783479-cee3-431e-a060-386bb21bcca0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:59:00.762369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1df8a9-f293-4d03-b0c9-d0de5d9a6ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:59:00.762970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df8a9-f293-4d03-b0c9-d0de5d9a6ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:00.763243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df8a9-f293-4d03-b0c9-d0de5d9a6ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:00.763507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1df8a9-f293-4d03-b0c9-d0de5d9a6ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:00.772122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1df8a9-f293-4d03-b0c9-d0de5d9a6ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:59:00.773542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1df8a9-f293-4d03-b0c9-d0de5d9a6ae0 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:59:00.783161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 772/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:59:02.925310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d74e-ad47-4b12-80a3-159ab37f9e4b None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:02.925834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d74e-ad47-4b12-80a3-159ab37f9e4b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:02.926150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849d74e-ad47-4b12-80a3-159ab37f9e4b None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:02.928459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:59:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:59:02.933689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414f5bb-5b5a-49de-9d42-a283a54b0f29 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:02.934140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414f5bb-5b5a-49de-9d42-a283a54b0f29 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:02.934350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414f5bb-5b5a-49de-9d42-a283a54b0f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:02.953495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9414f5bb-5b5a-49de-9d42-a283a54b0f29 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:59:02.954756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9414f5bb-5b5a-49de-9d42-a283a54b0f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:02.962503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:59:03.250212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d74e-ad47-4b12-80a3-159ab37f9e4b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:59:03.260925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849d74e-ad47-4b12-80a3-159ab37f9e4b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:59:03.264886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e2c4-cbfe-46cf-9698-9d9a7b6dc529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:59:03.265475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e2c4-cbfe-46cf-9698-9d9a7b6dc529 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:03.265748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e2c4-cbfe-46cf-9698-9d9a7b6dc529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:03.266040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e2c4-cbfe-46cf-9698-9d9a7b6dc529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:03.274834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e2c4-cbfe-46cf-9698-9d9a7b6dc529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:59:03.276224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e2c4-cbfe-46cf-9698-9d9a7b6dc529 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:59:03.285383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 760/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:59:07.005358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4558-828d-41ef-bf37-aad01e9f3bc2 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:07.005951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4558-828d-41ef-bf37-aad01e9f3bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:07.005951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc4558-828d-41ef-bf37-aad01e9f3bc2 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:07.007285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:59:07.013081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5895-db3b-494b-b329-dfdd8e53b2e6 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:07.013489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5895-db3b-494b-b329-dfdd8e53b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:07.013625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5895-db3b-494b-b329-dfdd8e53b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:07.021387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=81 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:59:07.035001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-325a5895-db3b-494b-b329-dfdd8e53b2e6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:59:07.036201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-325a5895-db3b-494b-b329-dfdd8e53b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:07.043907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 04:59:07.346687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc4558-828d-41ef-bf37-aad01e9f3bc2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:59:07.348473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-21cc4558-828d-41ef-bf37-aad01e9f3bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:07.413578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:59:07.426696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc4558-828d-41ef-bf37-aad01e9f3bc2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:59:07.430166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a5c8e2-223e-4381-8a2e-9ba965b52ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:59:07.430770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5c8e2-223e-4381-8a2e-9ba965b52ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:07.431044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5c8e2-223e-4381-8a2e-9ba965b52ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:07.431319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a5c8e2-223e-4381-8a2e-9ba965b52ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:07.440464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5c8e2-223e-4381-8a2e-9ba965b52ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:59:07.441879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a5c8e2-223e-4381-8a2e-9ba965b52ec2 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:59:07.451007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 775/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:59:09.583718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3189e-8074-48e0-adb2-ff0c57e987b8 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:09.584282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3189e-8074-48e0-adb2-ff0c57e987b8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:09.584282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3189e-8074-48e0-adb2-ff0c57e987b8 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:09.585588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 24 04:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 04:59:09.590700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5e0b6-1079-4d70-9c6a-64b08311e994 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:09.591088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5e0b6-1079-4d70-9c6a-64b08311e994 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:09.591250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5e0b6-1079-4d70-9c6a-64b08311e994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:09.611479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e7c5e0b6-1079-4d70-9c6a-64b08311e994 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 04:59:09.612668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e7c5e0b6-1079-4d70-9c6a-64b08311e994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:09.620399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 24 04:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 04:59:09.920955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3189e-8074-48e0-adb2-ff0c57e987b8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 04:59:09.922774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c4d3189e-8074-48e0-adb2-ff0c57e987b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 04:59:09.935625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3189e-8074-48e0-adb2-ff0c57e987b8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 04:59:09.939120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e57a0-b2f3-4c09-b718-fda124cc4a71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 04:59:09.939709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e57a0-b2f3-4c09-b718-fda124cc4a71 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 04:59:09.939982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e57a0-b2f3-4c09-b718-fda124cc4a71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 04:59:09.940250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e57a0-b2f3-4c09-b718-fda124cc4a71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 04:59:09.949028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e57a0-b2f3-4c09-b718-fda124cc4a71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 04:59:09.950476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e57a0-b2f3-4c09-b718-fda124cc4a71 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 04:59:09.960133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 763/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 04:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 04:59:19.606128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=11 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 04:59:19.956117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=8 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:34.350799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.352169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.352597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.379390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.381025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.394479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 777/1540] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.403986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.405009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.405505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.431401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.433004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.450430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 764/1541] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.459882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:34.472045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afceb173-94ed-4532-9951-b79db9fb569e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:34.481167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:34.481796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.482104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.482376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.483086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:34.483391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:34.485547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca50a52-d6dd-4554-acd6-3201244776f4 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:34.504042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 778/1542] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:34.513652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:34.525267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921cb53-24a6-48fe-b52b-7a55c204bcc9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:34.529928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03c4fe-310b-4f5a-b2b4-1ac0d2c2ee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:34.530617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03c4fe-310b-4f5a-b2b4-1ac0d2c2ee90 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.530890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03c4fe-310b-4f5a-b2b4-1ac0d2c2ee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.531208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03c4fe-310b-4f5a-b2b4-1ac0d2c2ee90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.531893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03c4fe-310b-4f5a-b2b4-1ac0d2c2ee90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:34.533847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03c4fe-310b-4f5a-b2b4-1ac0d2c2ee90 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:34.573336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.573717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.574011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.597020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 765/1543] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:34.600746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7f153-64a1-44c6-a41d-984674240505 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.601145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7f153-64a1-44c6-a41d-984674240505 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.601271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7f153-64a1-44c6-a41d-984674240505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.604599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.605956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.633493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-00c7f153-64a1-44c6-a41d-984674240505 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.634807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-00c7f153-64a1-44c6-a41d-984674240505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.722924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 779/1544] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:00:34.723783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:34.742821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a0714-f7b0-457a-aac6-03eeacf3d6e2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:34.747572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb8bfc-d32b-4f94-8c3f-dd97d61b607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:34.748196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb8bfc-d32b-4f94-8c3f-dd97d61b607a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.748474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb8bfc-d32b-4f94-8c3f-dd97d61b607a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.748747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb8bfc-d32b-4f94-8c3f-dd97d61b607a admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.749386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb8bfc-d32b-4f94-8c3f-dd97d61b607a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:34.752474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb8bfc-d32b-4f94-8c3f-dd97d61b607a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:34.784431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 780/1545] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/users => generated 289 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:34.785747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91d18-6a51-41b9-9349-307e086cc16c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.786568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91d18-6a51-41b9-9349-307e086cc16c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.786855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a91d18-6a51-41b9-9349-307e086cc16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.796672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 766/1546] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.798911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.799756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.800255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.816009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d4a91d18-6a51-41b9-9349-307e086cc16c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.820395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d4a91d18-6a51-41b9-9349-307e086cc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.832352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.833746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 781/1547] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.834307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.835158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5d532-2001-41c3-9f11-d92fa5d5ffd0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.835444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5d532-2001-41c3-9f11-d92fa5d5ffd0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.835715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5d532-2001-41c3-9f11-d92fa5d5ffd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.849375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 767/1548] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.850237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:34.859719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8468d-7900-4deb-bb9e-f2fad816cee1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:34.860092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-33d5d532-2001-41c3-9f11-d92fa5d5ffd0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.861300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-33d5d532-2001-41c3-9f11-d92fa5d5ffd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.864167 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc7fb5-a678-4af2-8e3d-55afbef87ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:34.864761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc7fb5-a678-4af2-8e3d-55afbef87ebd admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.865057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc7fb5-a678-4af2-8e3d-55afbef87ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.865322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc7fb5-a678-4af2-8e3d-55afbef87ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.865946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc7fb5-a678-4af2-8e3d-55afbef87ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:34.867597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc7fb5-a678-4af2-8e3d-55afbef87ebd admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:34.869312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 782/1549] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.870651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815a2d0-b22e-4a47-8e50-3928a51d1ee0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.870940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815a2d0-b22e-4a47-8e50-3928a51d1ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.871215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815a2d0-b22e-4a47-8e50-3928a51d1ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.878408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 768/1550] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:34.882449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9f12-f520-4f3f-9219-c520412d1442 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.882731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9f12-f520-4f3f-9219-c520412d1442 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.883000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a9f12-f520-4f3f-9219-c520412d1442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.891988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3815a2d0-b22e-4a47-8e50-3928a51d1ee0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.893198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3815a2d0-b22e-4a47-8e50-3928a51d1ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.901139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 783/1551] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.902326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22d43-733d-4d6a-b168-dff7d016bc52 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.902616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22d43-733d-4d6a-b168-dff7d016bc52 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.902793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4f3a9f12-f520-4f3f-9219-c520412d1442 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.902904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd22d43-733d-4d6a-b168-dff7d016bc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.903953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4f3a9f12-f520-4f3f-9219-c520412d1442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.912074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 769/1552] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.915184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e0112-047d-48b7-adc5-19be482b8112 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.915465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e0112-047d-48b7-adc5-19be482b8112 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.915738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e0112-047d-48b7-adc5-19be482b8112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.922172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1cd22d43-733d-4d6a-b168-dff7d016bc52 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.923538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1cd22d43-733d-4d6a-b168-dff7d016bc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.931451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 784/1553] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.932623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.932905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.933178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.934537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-1f7e0112-047d-48b7-adc5-19be482b8112 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.935736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1f7e0112-047d-48b7-adc5-19be482b8112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.944019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 770/1554] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.945253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.945531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.945801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.953594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.954810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.962681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 785/1555] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.963726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:34.973641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0e31b-8bac-43d1-9324-b096f8557080 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:34.973981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:34.975195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:34.978231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:34.978826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:34.979108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:34.979379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:34.980022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:34.980476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:34.982096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:34.983347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 771/1556] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:34.986856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.001772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 786/1557] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.001772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.002095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab1f9f-7d70-40d5-a93a-2f2ffb3fac6c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.002095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ff8fa-d0ef-425a-8630-e350afb5b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.002095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ff8fa-d0ef-425a-8630-e350afb5b14b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.002095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ff8fa-d0ef-425a-8630-e350afb5b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.002490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ff8fa-d0ef-425a-8630-e350afb5b14b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.002976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ff8fa-d0ef-425a-8630-e350afb5b14b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.004552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ff8fa-d0ef-425a-8630-e350afb5b14b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.011514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf983d-679f-4b47-a07e-90e43ed4816d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.018723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 772/1558] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:35.018723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.018723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.018723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.019882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.020587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.020868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.021142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.021847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:35.022181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.023800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933431a8-543f-4a93-83c7-df2194104ef1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.036968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:35.037364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 787/1559] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.038206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:35.038633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd09a3-f34c-4c2b-90f9-2f16d014a0ed None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.038915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd09a3-f34c-4c2b-90f9-2f16d014a0ed None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.039185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbd09a3-f34c-4c2b-90f9-2f16d014a0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.046872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 773/1560] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:35.047545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.059221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5d86d-09a2-4f34-8666-1f3b5474aa99 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.061979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5fbd09a3-f34c-4c2b-90f9-2f16d014a0ed None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:35.063716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5fbd09a3-f34c-4c2b-90f9-2f16d014a0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:35.064466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.065067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.065340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.065607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.066279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:35.066575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.067585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 24 05:00:35.067585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 05:00:35.067585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 05:00:35.068461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.071497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 788/1561] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:35.075315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.075612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.075895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.088153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 774/1562] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.088153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.095679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffc353-b147-427b-ac43-cffec577ea75 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.096831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:35.097807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:35.100943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.101591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.101942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.102269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.102870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:35.103166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.105480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.107480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 789/1563] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:35.108312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.118049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab20a0a-447b-46c2-8c94-ebab8b71fd66 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.121755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 775/1564] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.122656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.129933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.130603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.131006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.131294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.131996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.132798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5a2c5d-abfa-4623-ab5b-abf0dd453b0b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.133052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 05:00:35.133052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 05:00:35.133052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 05:00:35.134351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.137357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.138252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.138558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.138831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] PATH_INFO: `/v3/projects/66ec8bde739145dda504995dbea0914a/users/4b1b0448e7f54e3aaae49e411699269e/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.155009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66ec8bde739145dda504995dbea0914a, user_id=4b1b0448e7f54e3aaae49e411699269e, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.158281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.169748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 790/1565] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 393 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.170931 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.179776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 776/1566] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:35 2022] PUT /identity/v3/projects/66ec8bde739145dda504995dbea0914a/users/4b1b0448e7f54e3aaae49e411699269e/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:35.180606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.200172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:35.209752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:35.242279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8d9ba-a6ad-467f-ae65-19fd1e04aea3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.245360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19151a34-c748-4fb1-bde7-8753f1758585 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.246918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.247528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.247810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.248085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.248717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.250248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.251261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.251486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.251764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.252028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.252663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:35.252958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.254645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.266187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 777/1567] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.267064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.276886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68efc778-fa4d-403f-a6e1-c78a23902039 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.281754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.282462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.282813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.283163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.283864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:35.284203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.287596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 791/1568] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.287596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.287848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.298786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 778/1569] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.299578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.309175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9112e0-254f-4ad7-a592-b572e6876110 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.352433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68577ba-ddc2-4441-92dd-73d03942c6b0 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.358126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.358768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.359048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.359323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.360491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.362447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.364849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.365480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.365754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.366057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.366828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:35.366990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.368603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.379940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 779/1570] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:35.380673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.390677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7ea9c-0f6d-4d2a-bb90-4630d2e43d52 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.395957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f797a-6d4b-4bf1-ad81-20e84b92ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.396557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f797a-6d4b-4bf1-ad81-20e84b92ea15 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.396828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f797a-6d4b-4bf1-ad81-20e84b92ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.397096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f797a-6d4b-4bf1-ad81-20e84b92ea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.397719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f797a-6d4b-4bf1-ad81-20e84b92ea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.398220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 792/1571] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.399170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.400049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f797a-6d4b-4bf1-ad81-20e84b92ea15 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.432150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 780/1572] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.435569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.435855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.436135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.455082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:35.456285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:35.466497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238793d7-a521-40a7-9bde-8f5d46904c0b admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.513417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:35.514656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 781/1573] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:35.515607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.526220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bb02b-6b38-492f-a8e7-c2971425bfa1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.530967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.532134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.532421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.532696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.533373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.533974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.534621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.534899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.535170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.535511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.535803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.537768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.545039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:35.560528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 793/1574] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.561356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.567904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 782/1575] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.568809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.571891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec25b795-dde1-42fa-a6b1-f48daf359c44 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.627834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.628429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.628633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.628932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.629554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.630475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c57c13-9030-423c-9451-bd4de55f3a62 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.631498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.635117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.635719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.635999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.636274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.636901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.639097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.658254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 794/1576] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.658254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.666587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc80384a-9d42-48c6-9266-f899329451f8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.671401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.672030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.672556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.672830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.673181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 783/1577] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.673537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.674185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.675787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.710090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 795/1578] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.710090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.740141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfc30e-59e5-421a-b1aa-e2202c89e26e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.770406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b85e25-a339-41b5-ba54-ba8805c1720a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.775065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.775682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.775965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.776253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.776868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.778838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.796867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.797596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.797876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.798185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.798824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.799826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 05:00:35.799826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 05:00:35.799826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 05:00:35.800909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 796/1579] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.801055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.802924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.827607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 784/1580] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.827607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.835153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f96e2-ab4b-4c70-8e3e-883cddbdb463 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.839701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:35.839827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.840347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.840651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.840917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.841891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.843729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.849530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:35.855623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 785/1581] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:35.857037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.862664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9d7d9-3e09-49fe-a66a-8adc626cd5d8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.866477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.867067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.867346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.867543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ea9ba-2a99-4f62-ab25-59268ad2eeaf admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.867643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.872057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.872633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.872911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.873182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.873970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.876901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.877667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.878899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.888636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 797/1582] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:35.889438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.898912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05db52c-4da0-44d3-94c5-3fcbe4f2586a service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.902366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 786/1583] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.902366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.907689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.908303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.909612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.909612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.909612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.911107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.913361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab227b9-54af-40a2-8adf-204cb8e50165 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.918629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.919180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.919457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.919729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.920351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.925715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 798/1584] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:35.925875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.925990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.938230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef692be-cd0f-4bfc-855e-fef4412d23e9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.943240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.943956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.944312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.944658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.945362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.950527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 787/1585] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 282 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.950690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.950792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.961788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b026-42e5-4634-8167-d2e5537cca4d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.967558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.968383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.969282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.969572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.970296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.972247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:35.977628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 799/1586] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:35 2022] POST /identity/v3/users => generated 295 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:35.977628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.987102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26f933c-f9b7-4b29-927c-9e27e6efd185 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:35.992340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 788/1587] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:35.992340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:35.995605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:35.996484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:35.996834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:35.997233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:35.998023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:35.999935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.002194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e4879-fd47-4606-a578-814f2901ff35 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.007392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.008109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.008464 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.008835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.009529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.011281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.012665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 800/1588] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.013868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.024108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b45c487-9f4e-40e5-9e6a-a2bc627b3b94 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.027590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 789/1589] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.029618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.030257 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.030542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.030821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.031523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.031789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.033114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.043157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b7a59-9281-48b3-8fbe-9fa1eb3cc0e9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.046950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 801/1590] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.046950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.053845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.054639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.054827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.055194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.055844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.057452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.058300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731b09f-c119-4781-9768-2104aa7b435a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.066404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.067353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.068032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.068788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.069822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.074229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 790/1591] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.074229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.080006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.085860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff89c7b-d326-4359-b62a-ec16b4aef9de admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.092494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.094217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.094587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.095033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.096046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.100147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 802/1592] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.100147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.100384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.114365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 791/1593] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.114555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05369e5c-85f6-48d1-bf55-c9e4783e2eac admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.114653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.119269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.120549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.120935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.121292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.122040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.123758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab5730-f7cf-4229-9322-f3db5a4ea787 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.123989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.129509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.130070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.130352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.130625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.131389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.132979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.140538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 803/1594] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.140538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.147191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 792/1595] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.148183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.155311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419126b2-ed2a-4005-9155-a8069152f60b admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.164220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44638b-efff-473a-83cd-452c036850c8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.166260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.167091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.167442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.167725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] PATH_INFO: `/v3/projects/a18a3756ba2b460d977f738a115ebfe0/users/8cdd0da2c7bc4139bcd48ef36a2ee183/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.170419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.171309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.171762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.172938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.173732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.176809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.192395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18a3756ba2b460d977f738a115ebfe0, user_id=8cdd0da2c7bc4139bcd48ef36a2ee183, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.192395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 05:00:36.192395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 05:00:36.192395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 05:00:36.193181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 793/1596] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.194321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.194642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.207350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9d53-fa89-4428-ae44-0a8d80aef3b1 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.214619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.215413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.215671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.215946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] PATH_INFO: `/v3/projects/c8eb924c6ccf498e86777543349f2cf9/users/701e9898649546818ca5eef88e24041a/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.221551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 804/1597] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/a18a3756ba2b460d977f738a115ebfe0/users/8cdd0da2c7bc4139bcd48ef36a2ee183/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.221551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.243110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8eb924c6ccf498e86777543349f2cf9, user_id=701e9898649546818ca5eef88e24041a, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.246637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.275026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 794/1598] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/c8eb924c6ccf498e86777543349f2cf9/users/701e9898649546818ca5eef88e24041a/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.275026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.298763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201c893-8333-44b4-bdde-6db3e9221ed3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.350105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:36.373464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.374234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.374623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.374987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.376253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.378181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.383030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb905b-4d79-4d92-81bd-205292e480da admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.392993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.393216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 805/1599] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.393374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.393374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.393555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.393820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.393820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.393820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.404677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 795/1600] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:36.405408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.406427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a3b85-04d4-4cb3-8022-27e2378294c9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.411188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.412101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.412554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.412831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] PATH_INFO: `/v3/projects/653aa5e6a8ce469db4b8f20de0d5f5e5/users/28306a5f694744a2b1812c245eeba4c2/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.416093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555149d-eceb-43d6-a3f1-3e52659e7ce8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.421084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503330b-2566-4cdb-836e-c7c17caaa740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.421850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503330b-2566-4cdb-836e-c7c17caaa740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.422275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503330b-2566-4cdb-836e-c7c17caaa740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.422614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503330b-2566-4cdb-836e-c7c17caaa740 admin admin] PATH_INFO: `/v3/projects/87d5229e39fe4817b5187068d8acecc7/users/74a8cc19ab204424ab23c62915ac2bab/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.428485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=653aa5e6a8ce469db4b8f20de0d5f5e5, user_id=28306a5f694744a2b1812c245eeba4c2, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.431318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.441105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503330b-2566-4cdb-836e-c7c17caaa740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d5229e39fe4817b5187068d8acecc7, user_id=74a8cc19ab204424ab23c62915ac2bab, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.444150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503330b-2566-4cdb-836e-c7c17caaa740 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.450437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 806/1601] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/653aa5e6a8ce469db4b8f20de0d5f5e5/users/28306a5f694744a2b1812c245eeba4c2/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.451313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.466483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 796/1602] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/87d5229e39fe4817b5187068d8acecc7/users/74a8cc19ab204424ab23c62915ac2bab/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.470740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4970b09-67c4-4172-9155-1ba7bc3735fc None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.470740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4970b09-67c4-4172-9155-1ba7bc3735fc None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.470740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4970b09-67c4-4172-9155-1ba7bc3735fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.482205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:36.489133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c4970b09-67c4-4172-9155-1ba7bc3735fc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:36.491482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c4970b09-67c4-4172-9155-1ba7bc3735fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:36.524964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666725c-c197-4e35-8b82-72df579f82cd admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.567172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 797/1603] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:36.567172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.567172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.567172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.596971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] MFA Rules not processed for user `701e9898649546818ca5eef88e24041a`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:36.598289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:36.599790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.600710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.600995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.601322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] PATH_INFO: `/v3/projects/aa84dfa1cc764dc99b6e804e778c3479/users/27d1a24c9e15493195744bcbf598ac4a/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.623792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa84dfa1cc764dc99b6e804e778c3479, user_id=27d1a24c9e15493195744bcbf598ac4a, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.628152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.653937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 807/1604] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/aa84dfa1cc764dc99b6e804e778c3479/users/27d1a24c9e15493195744bcbf598ac4a/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.653937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.692538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 798/1605] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:36.694586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:36.710057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb08bf4-8696-410b-a769-51af39693b15 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.748014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:36.766761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2ceb90-fa66-4f68-a6fa-6835f1a5d1ef admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:36.773627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8bc786-7de1-4cc6-9e33-213da4683cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.774352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bc786-7de1-4cc6-9e33-213da4683cbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.774631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bc786-7de1-4cc6-9e33-213da4683cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.774905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8bc786-7de1-4cc6-9e33-213da4683cbe admin admin] PATH_INFO: `/v3/projects/378c46734a624fa28b2b10ad13a388d0/users/dbe7bdf12f0841349d0adac85089d00b/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.793057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bc786-7de1-4cc6-9e33-213da4683cbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=378c46734a624fa28b2b10ad13a388d0, user_id=dbe7bdf12f0841349d0adac85089d00b, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.796132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8bc786-7de1-4cc6-9e33-213da4683cbe admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.822080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 808/1606] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/378c46734a624fa28b2b10ad13a388d0/users/dbe7bdf12f0841349d0adac85089d00b/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.822080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.822080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.822080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.838184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff78c7-6073-441c-bef2-3ae1721c5a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:36.838947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff78c7-6073-441c-bef2-3ae1721c5a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.839208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff78c7-6073-441c-bef2-3ae1721c5a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.839493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff78c7-6073-441c-bef2-3ae1721c5a01 admin admin] PATH_INFO: `/v3/projects/c1aaf20f480a423ea5feca81585aeeed/users/0948754217fc4ff2ba0241fa35feaf90/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.859914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] MFA Rules not processed for user `28306a5f694744a2b1812c245eeba4c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:36.861724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:36.861964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff78c7-6073-441c-bef2-3ae1721c5a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1aaf20f480a423ea5feca81585aeeed, user_id=0948754217fc4ff2ba0241fa35feaf90, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:36.864649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff78c7-6073-441c-bef2-3ae1721c5a01 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:36.885204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 799/1607] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:36 2022] PUT /identity/v3/projects/c1aaf20f480a423ea5feca81585aeeed/users/0948754217fc4ff2ba0241fa35feaf90/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:36.887146 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131a574-33e0-49fb-a725-125a2f782206 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:36.887471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131a574-33e0-49fb-a725-125a2f782206 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:36.888369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131a574-33e0-49fb-a725-125a2f782206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:36.914653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3131a574-33e0-49fb-a725-125a2f782206 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:36.919049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3131a574-33e0-49fb-a725-125a2f782206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:36.939803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:36.943461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 809/1608] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:36.948891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.002325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 800/1609] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:37.004210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8aa4-90c2-4176-9b0c-cbd1c1a7bb8e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.004641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8aa4-90c2-4176-9b0c-cbd1c1a7bb8e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.005069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e8aa4-90c2-4176-9b0c-cbd1c1a7bb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.019986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9d428-722c-4260-89e5-ea2d70d7a2fd None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.029673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3b8e8aa4-90c2-4176-9b0c-cbd1c1a7bb8e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:37.031273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3b8e8aa4-90c2-4176-9b0c-cbd1c1a7bb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:37.048135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 801/1610] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:37.049717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190702a7-0d95-4e44-b916-ff242e40fb3d None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.050140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190702a7-0d95-4e44-b916-ff242e40fb3d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.050566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190702a7-0d95-4e44-b916-ff242e40fb3d None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.053188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 802/1611] 10.222.0.50 () {54 vars in 961 bytes} [Sat Sep 24 05:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:00:37.054544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.055736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.056131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.087965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] MFA Rules not processed for user `dbe7bdf12f0841349d0adac85089d00b`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:37.089554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:37.100286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f0113-ace4-4852-9051-f0e4f1d8ac08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.101209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0113-ace4-4852-9051-f0e4f1d8ac08 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.101490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0113-ace4-4852-9051-f0e4f1d8ac08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.101762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f0113-ace4-4852-9051-f0e4f1d8ac08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.114535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f0113-ace4-4852-9051-f0e4f1d8ac08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.116268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f0113-ace4-4852-9051-f0e4f1d8ac08 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.130233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 810/1612] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:37.131318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.131601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.132376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.161710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:37.163136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:37.176471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 811/1613] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:37.176471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.185490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9941a35-f0b0-4d2d-9fb5-95f22d50ea35 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.191738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d655a12a-4c44-47d7-a3f2-335b0906849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.192448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655a12a-4c44-47d7-a3f2-335b0906849d admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.192731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655a12a-4c44-47d7-a3f2-335b0906849d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.193002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d655a12a-4c44-47d7-a3f2-335b0906849d admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.193641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655a12a-4c44-47d7-a3f2-335b0906849d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.196522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d655a12a-4c44-47d7-a3f2-335b0906849d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.212182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:37.219620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 803/1614] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:37.220632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.236077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 812/1615] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:37.237820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.238290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.238653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.244107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a4bcf-5f23-4ef6-9a5f-e85e41a4110f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.322325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:37.325814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:37.337383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.338408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.338779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.339187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.478267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:37.486247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.488588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.493896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 813/1616] 10.222.0.50 () {58 vars in 1058 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:37.494885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.507188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8babb2-73d1-4458-9cab-669b397fad6b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.508813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 804/1617] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:37.509709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.511309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.512110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.512465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.512814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.519292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad25894-9089-4682-9f63-0d7b597321b2 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.523064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.523671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.523947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.524217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.529487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.530987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.535439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.537196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.549930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 805/1618] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:37.551082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.563614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e9d5c-b6b7-48e5-a6ad-869f24b1e827 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.566366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 814/1619] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:37.567872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.569944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.570642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.570976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.571256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.571892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.574075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.586866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6317bad-e6cb-4087-9cfe-f393979eeeb6 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.600558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.602096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.602430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.602713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.605817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.615016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.620944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 806/1620] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:37.622427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.672581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 815/1621] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:37.672875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.698535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918046c4-4b86-4413-8b98-1d91ff6ea680 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.769287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c7530-70f9-4380-963c-571983b54bd7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.776601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.777239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.777514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.777787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.834210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.834840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.835125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.835379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.836197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.838264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.855486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.857356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.874157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 807/1622] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:37.883615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.900607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 816/1623] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:37 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:37.901989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:37.905960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc1cf5-a8c4-4e31-9dd7-9488c6287b23 service nova] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.917734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ed0f2-e4bb-476a-b3b5-cde666738945 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:37.928114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.929039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.929976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.930510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.931445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.932056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:37.932737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:37.933020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:37.933346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:37.934461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:37.936437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:37.937640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.021540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 808/1624] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:38.030274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 817/1625] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:00:38.030274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.030629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.031591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb392aa6-d26f-4c2a-9a80-dfa2d436c36e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.031808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f37f8-b04d-4f51-b191-c38a8b355b7b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.031899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.031899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.031899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.031899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] PATH_INFO: `/v3/projects/7b4c234ef410476f85a29dbde06b47a6/users/70a2dfacd3dd47c39343ab0050c0147b/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.034819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.036934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.036934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.036934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] PATH_INFO: `/v3/projects/d5cda10485b24dc6ba0583c2fc7f7c28/users/998ecd9fbbe14e89a7ee0d563bf9f313/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.066582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4c234ef410476f85a29dbde06b47a6, user_id=70a2dfacd3dd47c39343ab0050c0147b, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.076614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.078726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5cda10485b24dc6ba0583c2fc7f7c28, user_id=998ecd9fbbe14e89a7ee0d563bf9f313, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.083262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:38.086636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.114350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 818/1626] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:37 2022] PUT /identity/v3/projects/7b4c234ef410476f85a29dbde06b47a6/users/70a2dfacd3dd47c39343ab0050c0147b/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:38.121302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.130337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 809/1627] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:37 2022] PUT /identity/v3/projects/d5cda10485b24dc6ba0583c2fc7f7c28/users/998ecd9fbbe14e89a7ee0d563bf9f313/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:38.133973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.194086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:38.298442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c12a8-6b7f-4c5b-b7c2-731e2e7b8d1e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.304238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.305173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.305654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.306094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.307696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.309817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.317195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d70c7-83f4-40c8-ae13-47920803609f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.323998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 819/1628] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:38.325002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.325814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.326336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.326758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.327812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.330114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.333104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.356718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 810/1629] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:38.361150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.366102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185cae-0db3-41b8-ae40-2fec20422ad2 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.373345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65deb2-3026-4437-b565-3e96b74e9d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.374299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65deb2-3026-4437-b565-3e96b74e9d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.374680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65deb2-3026-4437-b565-3e96b74e9d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.375058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65deb2-3026-4437-b565-3e96b74e9d72 admin admin] PATH_INFO: `/v3/projects/7b4c234ef410476f85a29dbde06b47a6/users/70a2dfacd3dd47c39343ab0050c0147b/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.378265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65deb2-3026-4437-b565-3e96b74e9d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4c234ef410476f85a29dbde06b47a6, user_id=70a2dfacd3dd47c39343ab0050c0147b, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.382643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17ab39-6d31-4052-9bd4-3f536f58bc43 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.383588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65deb2-3026-4437-b565-3e96b74e9d72 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.395367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2e7e6-ac6e-4d1b-ae82-5119a2cda21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.396432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2e7e6-ac6e-4d1b-ae82-5119a2cda21c admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.396858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2e7e6-ac6e-4d1b-ae82-5119a2cda21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.397376 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2e7e6-ac6e-4d1b-ae82-5119a2cda21c admin admin] PATH_INFO: `/v3/projects/d5cda10485b24dc6ba0583c2fc7f7c28/users/998ecd9fbbe14e89a7ee0d563bf9f313/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.399821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2e7e6-ac6e-4d1b-ae82-5119a2cda21c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5cda10485b24dc6ba0583c2fc7f7c28, user_id=998ecd9fbbe14e89a7ee0d563bf9f313, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.403218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2e7e6-ac6e-4d1b-ae82-5119a2cda21c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.410651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 820/1630] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:38 2022] PUT /identity/v3/projects/7b4c234ef410476f85a29dbde06b47a6/users/70a2dfacd3dd47c39343ab0050c0147b/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:38.412392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04667670-e63b-4e73-94f3-f7b036ec1c36 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.413010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04667670-e63b-4e73-94f3-f7b036ec1c36 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.413684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04667670-e63b-4e73-94f3-f7b036ec1c36 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.429910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 821/1632] 10.222.0.50 () {54 vars in 912 bytes} [Sat Sep 24 05:00:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:00:38.429910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140953-de17-47e4-9680-6d8dbc76c3b7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.429910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140953-de17-47e4-9680-6d8dbc76c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.429910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13140953-de17-47e4-9680-6d8dbc76c3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.434513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 811/1632] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:38 2022] PUT /identity/v3/projects/d5cda10485b24dc6ba0583c2fc7f7c28/users/998ecd9fbbe14e89a7ee0d563bf9f313/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:38.436490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.437058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.437901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.479524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-13140953-de17-47e4-9680-6d8dbc76c3b7 None None] MFA Rules not processed for user `70a2dfacd3dd47c39343ab0050c0147b`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:38.484057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-13140953-de17-47e4-9680-6d8dbc76c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:38.498445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:38.504231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:38.620632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 822/1633] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:38.624271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.624573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.624851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.646686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 812/1634] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:00:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:38.655323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.688873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] MFA Rules not processed for user `998ecd9fbbe14e89a7ee0d563bf9f313`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:38.690551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:38.747605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e32b2-4c82-470b-9a1d-23cd7b53953d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.758570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.764050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.765021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.765565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.766359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:38.788868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.789984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.808809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 813/1635] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:38.810430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.831074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ee75c-6699-4c81-a32c-b0d3f1a109de service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.838895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 823/1636] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:38.839608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.840623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.841223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.841631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.858408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.861875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.879080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 814/1637] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:38.890472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:38.915653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0454eb-1936-46df-b420-20d336fb80ac None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:38.925487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:38.926766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:38.927868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:38.928071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:38.951766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:38.954060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:38.974916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 824/1638] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:39.202762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:39.219979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb46ea-1867-4e7f-a511-8accb4243759 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:39.227755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31092b7e-f870-46e1-9872-b82d7e4e6086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:39.229818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092b7e-f870-46e1-9872-b82d7e4e6086 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:39.230156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092b7e-f870-46e1-9872-b82d7e4e6086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:39.231356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092b7e-f870-46e1-9872-b82d7e4e6086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:39.274771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:39.364585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31092b7e-f870-46e1-9872-b82d7e4e6086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:39.366950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31092b7e-f870-46e1-9872-b82d7e4e6086 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:39.386792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 815/1639] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:39.673447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:39.689714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897cf923-abde-4920-a8e9-8c83bb19b736 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:39.698896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc99c7f-12e5-4463-949f-a62ff3ea55f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:39.700852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99c7f-12e5-4463-949f-a62ff3ea55f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:39.701566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99c7f-12e5-4463-949f-a62ff3ea55f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:39.702208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99c7f-12e5-4463-949f-a62ff3ea55f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:39.821356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc99c7f-12e5-4463-949f-a62ff3ea55f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:39.825747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc99c7f-12e5-4463-949f-a62ff3ea55f7 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:39.851388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 825/1640] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:40.427510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe611d98-8e58-47a6-bd1a-78c0c3981949 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:40.428410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe611d98-8e58-47a6-bd1a-78c0c3981949 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:40.428852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe611d98-8e58-47a6-bd1a-78c0c3981949 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:40.430618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 816/1641] 10.222.0.50 () {54 vars in 961 bytes} [Sat Sep 24 05:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:00:40.442938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:40.443604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:40.444045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:40.475819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:40.478581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:40.494752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a43cf6-1852-4c4c-9029-0f300cf7ff27 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:40.495034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a43cf6-1852-4c4c-9029-0f300cf7ff27 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:40.495693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a43cf6-1852-4c4c-9029-0f300cf7ff27 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:40.496642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 817/1642] 10.222.0.50 () {54 vars in 912 bytes} [Sat Sep 24 05:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 05:00:40.502937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:40.503211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:40.503477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:40.508121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 826/1643] 10.222.0.50 () {58 vars in 1058 bytes} [Sat Sep 24 05:00:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:40.524875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:40.526181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:40.554445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:40.568165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 818/1644] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:00:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:40.572773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0731941-c2f0-4194-83eb-d4922994ff32 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:40.576587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f2ca3-79f7-4986-985b-1c22963533e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:40.577297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2ca3-79f7-4986-985b-1c22963533e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:40.577640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2ca3-79f7-4986-985b-1c22963533e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:40.578034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f2ca3-79f7-4986-985b-1c22963533e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:40.591315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2ca3-79f7-4986-985b-1c22963533e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:40.593028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f2ca3-79f7-4986-985b-1c22963533e4 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:40.618079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 827/1645] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:43.680151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:43.779834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7eb05c-fb0d-45fe-a4a6-60292bb9ce72 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:43.784839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:43.786343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:43.786862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:43.787532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] PATH_INFO: `/v3/users/dbe7bdf12f0841349d0adac85089d00b` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:43.789102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbe7bdf12f0841349d0adac85089d00b)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:43.791481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:43.851739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbe7bdf12f0841349d0adac85089d00b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:43.864090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbe7bdf12f0841349d0adac85089d00b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:43.865860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbe7bdf12f0841349d0adac85089d00b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:43.867454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbe7bdf12f0841349d0adac85089d00b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:43.880900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbe7bdf12f0841349d0adac85089d00b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:43.911370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 819/1646] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:00:43 2022] DELETE /identity/v3/users/dbe7bdf12f0841349d0adac85089d00b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:43.915623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:43.916133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:43.916265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:43.951152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:43.952471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:44.228966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 828/1647] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:00:44.285008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:44.290115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:44.456680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:44.509888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a69228-8034-4525-ae32-5bc47bf9e4c2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:44.531431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:44.532534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:44.532921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:44.540828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:44.556089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:44.557685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:44.583701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 820/1648] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:46.091037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:46.095432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:46.128665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bfe2c-7225-4f0c-8e73-f274a4b71967 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:46.140270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:46.141469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:46.141864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:46.142656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] PATH_INFO: `/v3/projects/378c46734a624fa28b2b10ad13a388d0` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:46.144091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=378c46734a624fa28b2b10ad13a388d0)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:46.146305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:46.272331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:46.274731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '378c46734a624fa28b2b10ad13a388d0', 'request_id': 'req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.291013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '378c46734a624fa28b2b10ad13a388d0', 'request_id': 'req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.304887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '378c46734a624fa28b2b10ad13a388d0', 'request_id': 'req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.336273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] The token cache is being invalidate because project 378c46734a624fa28b2b10ad13a388d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:00:46.336584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.337936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.342992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 829/1649] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:00:46 2022] DELETE /identity/v3/projects/378c46734a624fa28b2b10ad13a388d0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:46.356205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:46.375727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa23de-2c1c-44ad-abf0-05da93071552 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:46.497637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:46.498337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:46.498619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:46.498896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] PATH_INFO: `/v3/users/998ecd9fbbe14e89a7ee0d563bf9f313` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:46.500188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] RBAC: Authorizing `identity:delete_user(user_id=998ecd9fbbe14e89a7ee0d563bf9f313)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:46.503278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:46.602906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998ecd9fbbe14e89a7ee0d563bf9f313'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.615351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998ecd9fbbe14e89a7ee0d563bf9f313'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.616137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998ecd9fbbe14e89a7ee0d563bf9f313'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.616900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998ecd9fbbe14e89a7ee0d563bf9f313'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.631768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8105089e-276b-43cd-940c-8f5daec7740a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998ecd9fbbe14e89a7ee0d563bf9f313'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:46.678124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 821/1650] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:00:46 2022] DELETE /identity/v3/users/998ecd9fbbe14e89a7ee0d563bf9f313 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:48.134932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:48.136404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:48.156142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae6e77-ea1b-4d6e-ba88-b4acb6205a9e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:48.290204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:48.291611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:48.294179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:48.294822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] PATH_INFO: `/v3/projects/d5cda10485b24dc6ba0583c2fc7f7c28` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:48.297776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5cda10485b24dc6ba0583c2fc7f7c28)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:48.300597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:48.473480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5cda10485b24dc6ba0583c2fc7f7c28', 'request_id': 'req-006676bc-c469-4163-bb8f-487680e4b8cf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:48.485738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5cda10485b24dc6ba0583c2fc7f7c28', 'request_id': 'req-006676bc-c469-4163-bb8f-487680e4b8cf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:48.492205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5cda10485b24dc6ba0583c2fc7f7c28', 'request_id': 'req-006676bc-c469-4163-bb8f-487680e4b8cf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:48.523409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] The token cache is being invalidate because project d5cda10485b24dc6ba0583c2fc7f7c28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:00:48.523758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:48.524398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-006676bc-c469-4163-bb8f-487680e4b8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:48.526810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 830/1651] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:00:48 2022] DELETE /identity/v3/projects/d5cda10485b24dc6ba0583c2fc7f7c28 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:48.531495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:48.532010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:48.532010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:48.571570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:48.572837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:48.715494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:48.734284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 822/1652] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:48.742676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263b906-f644-4017-baae-43e23df59f29 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:48.742958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263b906-f644-4017-baae-43e23df59f29 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:48.743430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f263b906-f644-4017-baae-43e23df59f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:48.769458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f263b906-f644-4017-baae-43e23df59f29 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:48.779036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f263b906-f644-4017-baae-43e23df59f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:48.809226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 831/1653] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:48.810587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:48.838698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebea3ce-9883-4aaa-aec2-4fdedaf23652 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:48.851353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:48.851678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:48.851900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:48.875545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:48.878352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:48.878629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:48.878895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:48.881741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:00:48.884930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:48.889972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:48.937572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 823/1654] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:00:48.940501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:48.961274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] MFA Rules not processed for user `4b1b0448e7f54e3aaae49e411699269e`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:48.962596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:48.986843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b706a5-1ff7-4ef0-9b87-380e9a304789 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:49.004114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:49.004791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.005069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.006482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.007190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:49.009352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:49.100550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 824/1655] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:48 2022] POST /identity/v3/projects => generated 367 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:49.104337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:49.125831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:49.178876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 832/1656] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:49.195717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:49.199745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:49.274210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83684982-82b4-40d2-b8ab-0aa152369b37 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:49.290728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:49.291717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.292354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.292839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.293688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:49.302886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:49.352307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 825/1657] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:49 2022] POST /identity/v3/users => generated 284 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:49.355213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:49.389055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22cd71d-cd45-4e9e-9dc6-13377ee67518 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:49.408275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:49.409334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.409712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.410248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.411028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:49.412876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:49.419619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd90f00-a65b-44ed-a775-d8a466490043 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:49.450284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 826/1658] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:49.451691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:49.474712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cd9ee-240d-469e-927f-c5c909db192e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:49.491732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:49.492531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.493004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.493293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.494080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:49.496939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:49.533316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 827/1659] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:49.537386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:49.559787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f89829-36c1-419e-8256-e985294b3f1d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:49.573124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a521a80-883f-4ff0-a178-8e45daea4e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:49.574731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a521a80-883f-4ff0-a178-8e45daea4e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.576670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a521a80-883f-4ff0-a178-8e45daea4e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.579325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a521a80-883f-4ff0-a178-8e45daea4e8f admin admin] PATH_INFO: `/v3/projects/92c94dc5b06d4cdbb94682716dd3771f/users/d205927554f64deb87d32a5564d34b82/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.634853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:49.635506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.635783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.636062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.642514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a521a80-883f-4ff0-a178-8e45daea4e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c94dc5b06d4cdbb94682716dd3771f, user_id=d205927554f64deb87d32a5564d34b82, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:49.666547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a521a80-883f-4ff0-a178-8e45daea4e8f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:49.701384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 828/1660] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:49 2022] PUT /identity/v3/projects/92c94dc5b06d4cdbb94682716dd3771f/users/d205927554f64deb87d32a5564d34b82/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:49.723297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3a6c-c40a-453d-8e2d-f34c109442bf None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:49.723875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3a6c-c40a-453d-8e2d-f34c109442bf None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:49.724301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b3a6c-c40a-453d-8e2d-f34c109442bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:49.772902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-9c1b3a6c-c40a-453d-8e2d-f34c109442bf None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:49.774199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9c1b3a6c-c40a-453d-8e2d-f34c109442bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:49.836431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:49.838320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:49.866436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 833/1661] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:00:49.948710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 829/1662] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:50.037440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:50.039316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:50.094961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:50.174930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295eb9c-91ee-4134-8873-32b448132726 service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:50.289788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:50.290627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:50.290627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:50.290627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:50.316255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:50.317738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:50.339542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 834/1663] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:50.848564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d15f43-287b-4c53-8d4e-5d731d605889 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:50.849418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d15f43-287b-4c53-8d4e-5d731d605889 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:50.849835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d15f43-287b-4c53-8d4e-5d731d605889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:50.871080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:50.924040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d3d15f43-287b-4c53-8d4e-5d731d605889 None None] MFA Rules not processed for user `0948754217fc4ff2ba0241fa35feaf90`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:50.927014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d3d15f43-287b-4c53-8d4e-5d731d605889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:51.146474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:51.159070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 830/1664] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:51.282911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071db20-3b9f-4b90-9e5b-fbfbf3bfe515 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:51.302640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d0d7f-b91f-45fd-8587-46382fbe17c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:51.304235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d0d7f-b91f-45fd-8587-46382fbe17c1 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:51.304679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d0d7f-b91f-45fd-8587-46382fbe17c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:51.305720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d0d7f-b91f-45fd-8587-46382fbe17c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:51.330923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d0d7f-b91f-45fd-8587-46382fbe17c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:51.336061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d0d7f-b91f-45fd-8587-46382fbe17c1 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:51.378196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 835/1665] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:51.653023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:51.653312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:51.653611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:51.733953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] MFA Rules not processed for user `27d1a24c9e15493195744bcbf598ac4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:51.735816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:51.920825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 831/1666] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:51.932181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db01ed-1329-4263-9199-65afd4dc9049 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:51.932467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db01ed-1329-4263-9199-65afd4dc9049 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:51.932767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db01ed-1329-4263-9199-65afd4dc9049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:51.960341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d6db01ed-1329-4263-9199-65afd4dc9049 None None] MFA Rules not processed for user `27d1a24c9e15493195744bcbf598ac4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:51.961628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d6db01ed-1329-4263-9199-65afd4dc9049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:51.973298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 836/1667] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:51.985146 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:52.001538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55711069-d724-4ad0-bc36-3a557f702c3d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:52.009129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914b43f-c777-448d-a8a2-502c0167ac5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:52.010424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b43f-c777-448d-a8a2-502c0167ac5c service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:52.010671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b43f-c777-448d-a8a2-502c0167ac5c service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:52.010928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914b43f-c777-448d-a8a2-502c0167ac5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:52.027014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914b43f-c777-448d-a8a2-502c0167ac5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:52.028802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d914b43f-c777-448d-a8a2-502c0167ac5c service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:52.042520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 832/1668] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:53.357670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd0e30-1bdb-4cd0-a119-d011eaa405cb None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:53.358298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd0e30-1bdb-4cd0-a119-d011eaa405cb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:53.358298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd0e30-1bdb-4cd0-a119-d011eaa405cb None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:53.361232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 837/1669] 10.222.0.50 () {54 vars in 912 bytes} [Sat Sep 24 05:00:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:00:53.367891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676918a-a0d4-4117-9f72-2deda69da565 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:53.368402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676918a-a0d4-4117-9f72-2deda69da565 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:53.368488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2676918a-a0d4-4117-9f72-2deda69da565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:53.503477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2676918a-a0d4-4117-9f72-2deda69da565 None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:53.518971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2676918a-a0d4-4117-9f72-2deda69da565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:53.544278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 833/1670] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:53.591113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd0e30-1bdb-4cd0-a119-d011eaa405cb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:53.595021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fffd0e30-1bdb-4cd0-a119-d011eaa405cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:53.667045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd0e30-1bdb-4cd0-a119-d011eaa405cb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:53.671199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:53.672035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:53.672539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:53.673020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:53.688874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:53.690846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:53.724722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 838/1671] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:55.102373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:55.102957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:55.102957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:55.182439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] MFA Rules not processed for user `d205927554f64deb87d32a5564d34b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:55.199490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:55.371094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:55.463395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 834/1672] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:55.467319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:55.492794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7816aef8-59e8-4e5e-8eb8-7b88e31861ba service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:55.517195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cb545-badb-441d-b7f8-27f10944d06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:55.517832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb545-badb-441d-b7f8-27f10944d06a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:55.518144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb545-badb-441d-b7f8-27f10944d06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:55.518420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb545-badb-441d-b7f8-27f10944d06a admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:55.519066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cb545-badb-441d-b7f8-27f10944d06a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:55.521479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cb545-badb-441d-b7f8-27f10944d06a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:55.606236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 839/1673] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:00:55 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:55.607340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:55.623900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:55.629719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:55.630192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:55.652502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81f497f-94b6-46b4-a393-f0292a586d0e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:55.808316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] MFA Rules not processed for user `8cdd0da2c7bc4139bcd48ef36a2ee183`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:55.810146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:55.849307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:55.939099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:55.939741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:55.940050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:55.940346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:55.946811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:55.962069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:56.023642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 840/1674] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:00:56.025505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:56.030846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.038249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 835/1675] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:00:55 2022] POST /identity/v3/users => generated 293 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:00:56.041272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:56.047263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.124095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaa28d-f7f6-476f-89da-9dc0d7a9f545 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:56.134718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:56.135386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:56.135664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:56.140913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] PATH_INFO: `/v3/users/701e9898649546818ca5eef88e24041a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:56.155701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=701e9898649546818ca5eef88e24041a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:56.163232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: WARNING py.warnings [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 05:00:56.163232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: warnings.warn(msg) Sep 24 05:00:56.163232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: Sep 24 05:00:56.164838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:56.189205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce20afe-a9b5-4428-b604-d6e76af74ef4 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:56.201670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:56.202729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:56.202998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:56.203340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:56.205159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.220795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:56.222639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76ab90-f240-4fc6-9850-4d142c982bde service placement] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:56.288291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 836/1676] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:56.307443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:56.308129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:56.309460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:56.328715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701e9898649546818ca5eef88e24041a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:56.356694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701e9898649546818ca5eef88e24041a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:56.357244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701e9898649546818ca5eef88e24041a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:56.376196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] MFA Rules not processed for user `8cdd0da2c7bc4139bcd48ef36a2ee183`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:56.377462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.386377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701e9898649546818ca5eef88e24041a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:56.393669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701e9898649546818ca5eef88e24041a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:00:56.397790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 837/1677] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:56.398664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:56.457808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 841/1678] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:00:56 2022] DELETE /identity/v3/users/701e9898649546818ca5eef88e24041a => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:56.458998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:56.465602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.482695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac757c-5f69-45a7-8df1-40f47ada5cdd None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:56.670751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:56.695228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6286e9-6a10-407b-9a46-a30a8717e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:56.696227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6286e9-6a10-407b-9a46-a30a8717e249 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:56.702800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6286e9-6a10-407b-9a46-a30a8717e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:56.708241 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6286e9-6a10-407b-9a46-a30a8717e249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:56.709544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6286e9-6a10-407b-9a46-a30a8717e249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:56.709698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c7509-8ac4-4794-9e34-d8e20ac7de4c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:56.711549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6286e9-6a10-407b-9a46-a30a8717e249 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:56.733140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:56.733773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:56.734091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:56.734443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:56.737889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.774197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 838/1679] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:56.779193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:56.780518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:56.782874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:56.806322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:56.897632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:56.899621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:56.914548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:56.916360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.950306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 839/1680] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:56.953706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:56.962061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 842/1681] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:56.966705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:56.973989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:56.975137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:57.013306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90bc34-aa31-461b-ac21-29ee46b06d4f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:57.034559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:57.035968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:57.037853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:57.038894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] PATH_INFO: `/v3/projects/320e2bc6838f4f518440111b0f3e31c1/users/1dfeea504261439ebbf0f763b732f964/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:57.123513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=320e2bc6838f4f518440111b0f3e31c1, user_id=1dfeea504261439ebbf0f763b732f964, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:57.130842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:57.152852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b10221-2548-4005-9e45-9c5a5a6c326b service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:57.165252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 840/1682] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:56 2022] PUT /identity/v3/projects/320e2bc6838f4f518440111b0f3e31c1/users/1dfeea504261439ebbf0f763b732f964/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:57.177038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:57.342939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5baba2-97a1-4958-a0d2-a7f23bcb091a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:57.350089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5baba2-97a1-4958-a0d2-a7f23bcb091a service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:57.352107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5baba2-97a1-4958-a0d2-a7f23bcb091a service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:57.352107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5baba2-97a1-4958-a0d2-a7f23bcb091a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:57.476513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc575986-7980-466a-a4d8-1d9f15da1661 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:57.501141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:57.501776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:57.504669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:57.504964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:57.505827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:57.509224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:57.552196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 841/1683] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:00:57.553235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:57.587021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdc3e61-9f67-487c-8611-e9f59d148596 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:57.606848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2be35-d373-46ad-8045-ad90a8fa13ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:57.607561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5baba2-97a1-4958-a0d2-a7f23bcb091a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:57.609008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2be35-d373-46ad-8045-ad90a8fa13ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:57.609008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2be35-d373-46ad-8045-ad90a8fa13ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:57.609008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2be35-d373-46ad-8045-ad90a8fa13ec admin admin] PATH_INFO: `/v3/projects/320e2bc6838f4f518440111b0f3e31c1/users/1dfeea504261439ebbf0f763b732f964/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:57.610862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5baba2-97a1-4958-a0d2-a7f23bcb091a service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:57.617658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2be35-d373-46ad-8045-ad90a8fa13ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=320e2bc6838f4f518440111b0f3e31c1, user_id=1dfeea504261439ebbf0f763b732f964, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:57.629910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2be35-d373-46ad-8045-ad90a8fa13ec admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:57.646566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 843/1684] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:57.671355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 842/1685] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:00:57 2022] PUT /identity/v3/projects/320e2bc6838f4f518440111b0f3e31c1/users/1dfeea504261439ebbf0f763b732f964/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:00:58.077878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a3001-b48f-48ab-9e43-cfb173fc4e07 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:58.078462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a3001-b48f-48ab-9e43-cfb173fc4e07 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:58.079012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a3001-b48f-48ab-9e43-cfb173fc4e07 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:58.080674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 844/1686] 10.222.0.50 () {54 vars in 912 bytes} [Sat Sep 24 05:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:00:58.110033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3da30-84d1-48f9-89b9-9b583a3a2d94 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:58.113707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3da30-84d1-48f9-89b9-9b583a3a2d94 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:58.114293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3da30-84d1-48f9-89b9-9b583a3a2d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:58.173156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-30f3da30-84d1-48f9-89b9-9b583a3a2d94 None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:58.174471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-30f3da30-84d1-48f9-89b9-9b583a3a2d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:58.340263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 843/1687] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:00:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:58.403755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a3001-b48f-48ab-9e43-cfb173fc4e07 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:58.426614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a3001-b48f-48ab-9e43-cfb173fc4e07 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:58.441425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6ad13-ebaf-483b-bb2b-90e80f730602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:58.442082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6ad13-ebaf-483b-bb2b-90e80f730602 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:58.442538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6ad13-ebaf-483b-bb2b-90e80f730602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:58.442962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6ad13-ebaf-483b-bb2b-90e80f730602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:58.456473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6ad13-ebaf-483b-bb2b-90e80f730602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:58.458198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6ad13-ebaf-483b-bb2b-90e80f730602 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:58.476553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 845/1688] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:58.731889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:58.733955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:58.734802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:58.840202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] MFA Rules not processed for user `74a8cc19ab204424ab23c62915ac2bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:58.841467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:58.982958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:58.983552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:58.984068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:58.987614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:59.043256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 844/1689] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:59.044204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] MFA Rules not processed for user `4b1b0448e7f54e3aaae49e411699269e`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:00:59.047116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:59.078793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:00:59.241858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 846/1690] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:00:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:00:59.377562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:59.379274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:59.540579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:00:59.541832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803cbf53-91e7-4e5e-8c30-ef0d05078b66 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:59.544031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:00:59.557744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b440697-b364-4270-bb93-330f59208b4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:59.558603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b440697-b364-4270-bb93-330f59208b4c service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:59.559848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b440697-b364-4270-bb93-330f59208b4c service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:59.560280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b440697-b364-4270-bb93-330f59208b4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:59.603471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b440697-b364-4270-bb93-330f59208b4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:59.605103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b440697-b364-4270-bb93-330f59208b4c service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:00:59.662775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 845/1691] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:00:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:00:59.905528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dcd13b-03c7-4f98-8bd4-1823d3ad9609 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:00:59.917881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:00:59.918671 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:00:59.918958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:00:59.919236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] PATH_INFO: `/v3/projects/c8eb924c6ccf498e86777543349f2cf9` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:00:59.923337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8eb924c6ccf498e86777543349f2cf9)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:00:59.939689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:00.107532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8eb924c6ccf498e86777543349f2cf9', 'request_id': 'req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:00.116641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8eb924c6ccf498e86777543349f2cf9', 'request_id': 'req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:00.123555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8eb924c6ccf498e86777543349f2cf9', 'request_id': 'req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:00.155029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] The token cache is being invalidate because project c8eb924c6ccf498e86777543349f2cf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:00.155623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:00.157029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c157cbcb-46f6-4b1e-9ed9-0d7646f77135 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:00.163908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 847/1692] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:00:59 2022] DELETE /identity/v3/projects/c8eb924c6ccf498e86777543349f2cf9 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:00.165512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cbbbc-1913-4477-b7a3-f69e57f33d10 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:00.166015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cbbbc-1913-4477-b7a3-f69e57f33d10 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:00.166457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cbbbc-1913-4477-b7a3-f69e57f33d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:00.207419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-009cbbbc-1913-4477-b7a3-f69e57f33d10 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:00.216196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-009cbbbc-1913-4477-b7a3-f69e57f33d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:00.398946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:00.399692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:00.400620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:00.434048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 846/1693] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:00.447553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:00.447844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:00.454107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:00.501214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:00.502598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:00.527216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 847/1694] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:00.531614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:00.545625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:00.548401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:00.550591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a47561c-ad34-480e-93ae-fda9bb62d66a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:00.564272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:00.564905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:00.565187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:00.565500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:00.566498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:00.566924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:00.571190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3ca2-67b3-44d1-bf21-16517a831bd0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:00.599104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 848/1695] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:01:00.603332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:00.605763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:00.606250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:00.609674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:00.642358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] MFA Rules not processed for user `8cdd0da2c7bc4139bcd48ef36a2ee183`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:00.644729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:00.743655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 848/1696] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:01:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:00.745606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:00.792029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:00.803687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d67e6-057d-406d-8f54-7da1fbb26057 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:00.818501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 849/1697] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:00.839434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:00.840078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:00.843027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:00.843478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:00.847891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:00.853483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:00.967112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 849/1698] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:00 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:00.968314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:00.985238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46160347-7bb7-4e62-9ed0-438f22dd4206 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:01.074379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:01.076203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:01.162987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:01.163943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.164373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.164655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.165620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:01.167605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:01.214561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 850/1699] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:00 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:01.219113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:01.239231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5145a0-8161-4bb3-aa71-25768a6d9d1f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:01.248027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:01.248647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.248932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.249209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.252183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:01.254011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:01.271920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 851/1700] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:01.273032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:01.311651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9da8d-a448-4d37-8229-153f9c931604 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:01.330819 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:01.332008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.332289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.332567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.333222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:01.339882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:01.377389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 852/1701] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:01.378420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:01.400514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b722-1a00-4bf2-ab99-abe045db76d7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:01.414976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174a955-78e4-4af9-b76b-90058ec3fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:01.417473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a955-78e4-4af9-b76b-90058ec3fee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.418510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a955-78e4-4af9-b76b-90058ec3fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.419230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a955-78e4-4af9-b76b-90058ec3fee8 admin admin] PATH_INFO: `/v3/projects/8b0a47b2b3704d5180258eb3c5b7cfef/users/a3aa1a166c4f4108b76f2fd174117694/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.439834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174a955-78e4-4af9-b76b-90058ec3fee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0a47b2b3704d5180258eb3c5b7cfef, user_id=a3aa1a166c4f4108b76f2fd174117694, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:01.444700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174a955-78e4-4af9-b76b-90058ec3fee8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:01.473411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 853/1702] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:01 2022] PUT /identity/v3/projects/8b0a47b2b3704d5180258eb3c5b7cfef/users/a3aa1a166c4f4108b76f2fd174117694/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:01.474157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99ef02-d255-4c56-913b-e2b460a4e045 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:01.487791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2693b-3723-482f-88e0-e5f1fcdd64a8 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.489057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2693b-3723-482f-88e0-e5f1fcdd64a8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.489800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2693b-3723-482f-88e0-e5f1fcdd64a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.534071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d5a2693b-3723-482f-88e0-e5f1fcdd64a8 None None] MFA Rules not processed for user `a3aa1a166c4f4108b76f2fd174117694`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:01.535912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d5a2693b-3723-482f-88e0-e5f1fcdd64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:01.658479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5db235-014d-4e59-beec-bffe71e8de35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:01.660108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5db235-014d-4e59-beec-bffe71e8de35 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.661603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5db235-014d-4e59-beec-bffe71e8de35 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.662820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5db235-014d-4e59-beec-bffe71e8de35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.727880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 854/1703] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:01.751300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:01.829900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:01.830398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:01.830677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:01.831424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5db235-014d-4e59-beec-bffe71e8de35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:01.833130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5db235-014d-4e59-beec-bffe71e8de35 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:01.858850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] MFA Rules not processed for user `0948754217fc4ff2ba0241fa35feaf90`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:01.878932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:01.881165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 850/1704] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:01:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:01:01.970017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:02.112297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 855/1705] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:02.160392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d1d39-70e5-4fbb-ae70-6c62cc600878 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:02.160974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d1d39-70e5-4fbb-ae70-6c62cc600878 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:02.161608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d1d39-70e5-4fbb-ae70-6c62cc600878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:02.246306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-cd8d1d39-70e5-4fbb-ae70-6c62cc600878 None None] MFA Rules not processed for user `1dfeea504261439ebbf0f763b732f964`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:02.247754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cd8d1d39-70e5-4fbb-ae70-6c62cc600878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:02.458760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 851/1706] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:02.477677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:02.479399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:02.662900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c20238-15ce-491a-a4ed-a779bf31c6de None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:02.675097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:02.676185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:02.676785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:02.677378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:02.681978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:02.850232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:02.851721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:02.896460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 856/1707] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:03.053557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:03.054200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:03.054200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:03.133489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:03.134805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:03.288954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 852/1708] 10.222.0.16 () {58 vars in 1015 bytes} [Sat Sep 24 05:01:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:03.456759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:03.458615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:03.540710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:03.542704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:03.565078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd75f33-becb-4ca3-93a1-16ab8a34cd62 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:03.575194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:03.576463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:03.576952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:03.577459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:03.599066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:03.602552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:03.624508 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 853/1709] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:03.643386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5953c610-84fb-45a5-93dc-cb0c93bb5052 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:03.653903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6597776-fee1-43ba-8a19-9ec9abda4b78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:03.654721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597776-fee1-43ba-8a19-9ec9abda4b78 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:03.655235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597776-fee1-43ba-8a19-9ec9abda4b78 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:03.655641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6597776-fee1-43ba-8a19-9ec9abda4b78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:03.668527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6597776-fee1-43ba-8a19-9ec9abda4b78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:03.670431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6597776-fee1-43ba-8a19-9ec9abda4b78 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:03.699598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 857/1710] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:06.408191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:06.409892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:06.445480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca390e-9c46-4134-99a6-c70230ea5f4f service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:06.451212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28629e6-41fe-41c8-8c4f-00c1589dcc0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:06.452097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28629e6-41fe-41c8-8c4f-00c1589dcc0a service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:06.452521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28629e6-41fe-41c8-8c4f-00c1589dcc0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:06.453003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28629e6-41fe-41c8-8c4f-00c1589dcc0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:06.481071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28629e6-41fe-41c8-8c4f-00c1589dcc0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:06.482713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28629e6-41fe-41c8-8c4f-00c1589dcc0a service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:06.529061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 854/1711] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:07.907162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd934f8c-7d7a-4586-bfb8-54779ddd0082 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:07.907619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd934f8c-7d7a-4586-bfb8-54779ddd0082 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:07.907619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd934f8c-7d7a-4586-bfb8-54779ddd0082 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:07.909261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 858/1712] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:01:08.159226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:08.160230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:08.160793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:08.208233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:08.209497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:08.229258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 855/1713] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:01:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:08.882778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd934f8c-7d7a-4586-bfb8-54779ddd0082 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:08.898052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:08.900463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:08.954245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:08.965638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4839c69-3fe0-4d00-a2c7-42cef513de24 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:09.000218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9390c8-07ee-48ab-a71c-5d4b46a6c17f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:09.002620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9390c8-07ee-48ab-a71c-5d4b46a6c17f service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:09.008662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9390c8-07ee-48ab-a71c-5d4b46a6c17f service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:09.009154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9390c8-07ee-48ab-a71c-5d4b46a6c17f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:09.063808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9390c8-07ee-48ab-a71c-5d4b46a6c17f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:09.073913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9390c8-07ee-48ab-a71c-5d4b46a6c17f service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:09.128118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 856/1714] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:09.135938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd934f8c-7d7a-4586-bfb8-54779ddd0082 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:09.142745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:09.143947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:09.144551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:09.145269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] PATH_INFO: `/v3/users/a3aa1a166c4f4108b76f2fd174117694` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:09.147094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3aa1a166c4f4108b76f2fd174117694)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:09.149688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:09.283094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3aa1a166c4f4108b76f2fd174117694'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:09.309212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3aa1a166c4f4108b76f2fd174117694'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:09.312141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3aa1a166c4f4108b76f2fd174117694'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:09.325871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3aa1a166c4f4108b76f2fd174117694'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:09.391419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3aa1a166c4f4108b76f2fd174117694'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:09.455703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 859/1715] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:08 2022] DELETE /identity/v3/users/a3aa1a166c4f4108b76f2fd174117694 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:09.457925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848bf6-a721-455d-809c-5f8e74d80d67 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:09.458545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848bf6-a721-455d-809c-5f8e74d80d67 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:09.458545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848bf6-a721-455d-809c-5f8e74d80d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:09.555520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ee848bf6-a721-455d-809c-5f8e74d80d67 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:09.561321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ee848bf6-a721-455d-809c-5f8e74d80d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:09.590646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:09.756361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 857/1716] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:10.633663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:10.636707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:10.927409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8a02b-ffba-4491-8e4a-a091adde09ed admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:10.932278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17637c96-5304-4637-b4c9-753130ec08f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:10.933017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17637c96-5304-4637-b4c9-753130ec08f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:10.933158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17637c96-5304-4637-b4c9-753130ec08f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:10.934124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17637c96-5304-4637-b4c9-753130ec08f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:10.952464 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17637c96-5304-4637-b4c9-753130ec08f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:10.957817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17637c96-5304-4637-b4c9-753130ec08f8 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:10.990867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 860/1717] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:11.880029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aee3a-aa6d-45ae-81c7-230079d12f9d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:11.880628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aee3a-aa6d-45ae-81c7-230079d12f9d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:11.880628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961aee3a-aa6d-45ae-81c7-230079d12f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:11.931172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-961aee3a-aa6d-45ae-81c7-230079d12f9d None None] MFA Rules not processed for user `74a8cc19ab204424ab23c62915ac2bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:11.935522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-961aee3a-aa6d-45ae-81c7-230079d12f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:12.172151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 858/1718] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:12.408974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a077a5-0bd9-4d57-a8bc-01cbdff4bfab None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:12.409482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a077a5-0bd9-4d57-a8bc-01cbdff4bfab None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:12.409564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a077a5-0bd9-4d57-a8bc-01cbdff4bfab None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:12.412002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 861/1719] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:01:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:01:12.556342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3d735-258c-4e25-b20d-f732484abdb2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:12.556933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3d735-258c-4e25-b20d-f732484abdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:12.557515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3d735-258c-4e25-b20d-f732484abdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:12.703414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-87d3d735-258c-4e25-b20d-f732484abdb2 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:12.706723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-87d3d735-258c-4e25-b20d-f732484abdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:12.952251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 859/1720] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:01:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:13.392991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a077a5-0bd9-4d57-a8bc-01cbdff4bfab None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:13.394760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b6a077a5-0bd9-4d57-a8bc-01cbdff4bfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:13.424593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a077a5-0bd9-4d57-a8bc-01cbdff4bfab None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:13.449857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:13.450817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:13.451277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:13.451692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] PATH_INFO: `/v3/projects/8b0a47b2b3704d5180258eb3c5b7cfef` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:13.453363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0a47b2b3704d5180258eb3c5b7cfef)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:13.454584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: WARNING py.warnings [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 24 05:01:13.454584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: warnings.warn(msg) Sep 24 05:01:13.454584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: Sep 24 05:01:13.456307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:13.537708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:13.584352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0a47b2b3704d5180258eb3c5b7cfef', 'request_id': 'req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:13.594605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0a47b2b3704d5180258eb3c5b7cfef', 'request_id': 'req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:13.639374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0a47b2b3704d5180258eb3c5b7cfef', 'request_id': 'req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:13.677330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] The token cache is being invalidate because project 8b0a47b2b3704d5180258eb3c5b7cfef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:13.678103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:13.678939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7f1d4e28-ef0e-457d-a2d9-9eb3ef919491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:13.681058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 862/1721] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:13 2022] DELETE /identity/v3/projects/8b0a47b2b3704d5180258eb3c5b7cfef => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:13.686270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:13.686561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:13.686836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:13.716202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:13.718048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:13.779714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:13.799216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 860/1722] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:13.806551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:13.806843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:13.807100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:13.829301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:13.830491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:13.838589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 863/1723] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:13.844185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:13.855005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a156c-5a56-4886-825a-b5f930e6c570 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:13.862155 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:13.863172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:13.863630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:13.864088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:13.865062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:13.865596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:13.867688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:13.884658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 861/1724] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:01:13.889443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:13.899362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f626b-6da3-4835-9d18-a8b6dff040ec None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:13.905020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:13.905778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:13.906927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:13.907411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:13.908373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:13.910829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:13.944521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 864/1725] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:13 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:13.948963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:13.965832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53651f0-ddd5-44ee-ab7c-f26f8af46b8c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.061612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.062838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.063113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.063384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.064030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.074643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.075450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.077661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:14.108026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 862/1726] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:13 2022] POST /identity/v3/users => generated 282 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:14.112609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.128732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cfc56-b57c-4359-8309-7e6312556a5e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.135681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.136491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.137478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.137908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.139265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.141809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.173926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 863/1727] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:14.175346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.175764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343d61c-134c-4636-a76b-56c62548f574 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.183700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.184343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.184630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.184906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.187805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:14.188213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24374e-dfe6-4545-bbe7-935225ece2b2 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.194356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.195475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.195882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.196305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.197190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.199288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.204130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.205977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e56db7-7d39-49c1-ac94-868846c11bd2 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.214727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 864/1728] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:14.218639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.230250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b4cfbc-fd39-4fbb-af4e-038e482314ce admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.236858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.237607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.237883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.238192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] PATH_INFO: `/v3/projects/b47e04abe52c4c39b7d4b36659561f53/users/73e3cb5faba641b5b8961e2b750a5522/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.252078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 865/1729] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:14.279601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47e04abe52c4c39b7d4b36659561f53, user_id=73e3cb5faba641b5b8961e2b750a5522, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.283198 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.313401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 865/1730] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:14 2022] PUT /identity/v3/projects/b47e04abe52c4c39b7d4b36659561f53/users/73e3cb5faba641b5b8961e2b750a5522/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:14.318863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.320720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.321151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.380354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] MFA Rules not processed for user `73e3cb5faba641b5b8961e2b750a5522`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:14.382215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:14.486175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 866/1731] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:14.487341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.577329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52cb23-a83f-49e0-995b-04a466b249bf admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.584819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.585576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.585943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.586340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.587576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.589707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.638043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 866/1732] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:14 2022] POST /identity/v3/projects => generated 363 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:14.640571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.646840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:14.653260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9afa83-d8b1-46c5-b7b4-6688e76b81f6 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.728644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.729270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.729552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.729825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.731002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.733843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.770061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 867/1733] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:14 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:14.772852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.789538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2651e0-75e0-4a81-8c74-d896e6388813 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.794387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.795000 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.795277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.795550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.796188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.797902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.800364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:14.816926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 867/1734] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:14.817560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.828134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909d4bd-ea10-4c1c-bc88-5e74f6665341 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.834551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.835292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.835647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.835985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] PATH_INFO: `/v3/projects/b4202362f8fc4747b9a49a6a6b6f28e6/users/a71c97a02837477a9ced13a16d4c91ec/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.858992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4202362f8fc4747b9a49a6a6b6f28e6, user_id=a71c97a02837477a9ced13a16d4c91ec, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.862704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.877755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 868/1735] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:14 2022] PUT /identity/v3/projects/b4202362f8fc4747b9a49a6a6b6f28e6/users/a71c97a02837477a9ced13a16d4c91ec/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:14.881432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.891591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db64b3-d545-4b1a-95dc-7a09fb7cd89a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.947876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b6e49e-61e5-4ef0-8436-390cee81ba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.948409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6e49e-61e5-4ef0-8436-390cee81ba90 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.948688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6e49e-61e5-4ef0-8436-390cee81ba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.948955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b6e49e-61e5-4ef0-8436-390cee81ba90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.949585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6e49e-61e5-4ef0-8436-390cee81ba90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.951195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b6e49e-61e5-4ef0-8436-390cee81ba90 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.961526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 868/1736] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:14.965625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:14.976473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57896645-0612-4949-b882-e036eaec95bd admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:14.980972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:14.981610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:14.981882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:14.982178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] PATH_INFO: `/v3/projects/b4202362f8fc4747b9a49a6a6b6f28e6/users/a71c97a02837477a9ced13a16d4c91ec/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:14.984553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4202362f8fc4747b9a49a6a6b6f28e6, user_id=a71c97a02837477a9ced13a16d4c91ec, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:14.987197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:14.997518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 869/1737] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:14 2022] PUT /identity/v3/projects/b4202362f8fc4747b9a49a6a6b6f28e6/users/a71c97a02837477a9ced13a16d4c91ec/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:15.009051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:15.009533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:15.009635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:15.038612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] MFA Rules not processed for user `a71c97a02837477a9ced13a16d4c91ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:15.039826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:15.111185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 869/1738] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:15.123113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:15.124959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:15.183661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9b45b-8e65-4726-b0f0-9c1dc7116f26 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:15.187908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e530e183-feb9-41f8-8ecc-75906c8b3357 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:15.188567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530e183-feb9-41f8-8ecc-75906c8b3357 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:15.188953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530e183-feb9-41f8-8ecc-75906c8b3357 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:15.189162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e530e183-feb9-41f8-8ecc-75906c8b3357 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:15.199179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530e183-feb9-41f8-8ecc-75906c8b3357 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:15.200554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e530e183-feb9-41f8-8ecc-75906c8b3357 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:15.210590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 870/1739] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:15.297835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:15.299558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:15.311473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965ad9b-f360-40ea-98ae-20eff654b07d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:15.315673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1f4e2-b09f-4242-a235-0667bcfeb80b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:15.316279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1f4e2-b09f-4242-a235-0667bcfeb80b service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:15.316558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1f4e2-b09f-4242-a235-0667bcfeb80b service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:15.316832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1f4e2-b09f-4242-a235-0667bcfeb80b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:15.372719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1f4e2-b09f-4242-a235-0667bcfeb80b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:15.374177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1f4e2-b09f-4242-a235-0667bcfeb80b service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:15.382945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 870/1740] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:25.206373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:25.379130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:33.181421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666f310-e8ad-4a8e-9d8d-a21b0cd59b63 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:33.182975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666f310-e8ad-4a8e-9d8d-a21b0cd59b63 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:33.183260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666f310-e8ad-4a8e-9d8d-a21b0cd59b63 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:33.184816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 871/1741] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:01:33.196889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:33.197488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:33.197488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:33.220807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:33.222750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:33.315418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 871/1742] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:01:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:33.329172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666f310-e8ad-4a8e-9d8d-a21b0cd59b63 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:33.347684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666f310-e8ad-4a8e-9d8d-a21b0cd59b63 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:33.354694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06f70a-d6df-441e-a77b-52eb8a601e33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:33.355402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06f70a-d6df-441e-a77b-52eb8a601e33 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:33.355780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06f70a-d6df-441e-a77b-52eb8a601e33 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:33.356128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06f70a-d6df-441e-a77b-52eb8a601e33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:33.365314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06f70a-d6df-441e-a77b-52eb8a601e33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:33.367055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06f70a-d6df-441e-a77b-52eb8a601e33 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:33.378866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 872/1743] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:34.130685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:34.227330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fff46c-d09f-429e-aca3-c75de01a75b5 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:34.238605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:34.239704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:34.240366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:34.240827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] PATH_INFO: `/v3/users/73e3cb5faba641b5b8961e2b750a5522` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:34.242623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e3cb5faba641b5b8961e2b750a5522)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:34.248436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:34.331915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e3cb5faba641b5b8961e2b750a5522'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:34.345070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e3cb5faba641b5b8961e2b750a5522'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:34.345718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e3cb5faba641b5b8961e2b750a5522'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:34.359434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e3cb5faba641b5b8961e2b750a5522'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:34.360541 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e3cb5faba641b5b8961e2b750a5522'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:34.391235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 872/1744] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:34 2022] DELETE /identity/v3/users/73e3cb5faba641b5b8961e2b750a5522 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:34.395697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:34.396887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:34.397542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:34.465407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:34.466618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:34.571528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 873/1745] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:34.592405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:34.594602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:34.614748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:34.669638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc92caf-8ca1-4023-84be-8f50cfdd8e8a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:34.675413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:34.676072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:34.676373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:34.676754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:34.699900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:34.701821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:34.717146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 873/1746] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:35.619482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:35.621111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:35.638208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928657e9-e07e-4ddb-af1c-43faadb6edee None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:35.646151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:35.647982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:35.648483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:35.648912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] PATH_INFO: `/v3/projects/b47e04abe52c4c39b7d4b36659561f53` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:35.650689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47e04abe52c4c39b7d4b36659561f53)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:35.653192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:35.725683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47e04abe52c4c39b7d4b36659561f53', 'request_id': 'req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.747297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47e04abe52c4c39b7d4b36659561f53', 'request_id': 'req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.758398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47e04abe52c4c39b7d4b36659561f53', 'request_id': 'req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.774027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] The token cache is being invalidate because project b47e04abe52c4c39b7d4b36659561f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:35.774027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.774685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-37cfe964-a4ea-4e61-8c3e-47c925cc47cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.776199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 874/1747] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:35 2022] DELETE /identity/v3/projects/b47e04abe52c4c39b7d4b36659561f53 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:35.780385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:35.790663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4acdb-a232-422e-a134-2a2fa044b399 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:35.858386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:35.859109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:35.859397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:35.859670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] PATH_INFO: `/v3/users/a71c97a02837477a9ced13a16d4c91ec` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:35.860807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71c97a02837477a9ced13a16d4c91ec)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:35.865472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:35.917196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71c97a02837477a9ced13a16d4c91ec'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.928837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71c97a02837477a9ced13a16d4c91ec'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.930396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71c97a02837477a9ced13a16d4c91ec'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.939815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71c97a02837477a9ced13a16d4c91ec'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.940600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ae0b15b-42e3-416f-b678-683dd355137d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71c97a02837477a9ced13a16d4c91ec'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:35.961478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 874/1748] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:35 2022] DELETE /identity/v3/users/a71c97a02837477a9ced13a16d4c91ec => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:36.786791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45570f4-b1e7-4e3d-8bbf-48823ece0557 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:36.787302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45570f4-b1e7-4e3d-8bbf-48823ece0557 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:36.787399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45570f4-b1e7-4e3d-8bbf-48823ece0557 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:36.788702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 875/1749] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:01:36.859314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:36.859846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:36.860135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:36.882068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:36.883304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:36.919265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:36.972832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 875/1750] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:36.990996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45570f4-b1e7-4e3d-8bbf-48823ece0557 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:36.992690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f45570f4-b1e7-4e3d-8bbf-48823ece0557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:37.027665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.033483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:37.068913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8971b9a-e5ad-49e1-8728-eb3cacfe888f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.083914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9f751-0178-48ac-8723-222fbd9d6982 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.085210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9f751-0178-48ac-8723-222fbd9d6982 service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.085557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9f751-0178-48ac-8723-222fbd9d6982 service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.085838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9f751-0178-48ac-8723-222fbd9d6982 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.090894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45570f4-b1e7-4e3d-8bbf-48823ece0557 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.098217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.099014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.099355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.099699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] PATH_INFO: `/v3/projects/b4202362f8fc4747b9a49a6a6b6f28e6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.101267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4202362f8fc4747b9a49a6a6b6f28e6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.103582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.105428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9f751-0178-48ac-8723-222fbd9d6982 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.111818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9f751-0178-48ac-8723-222fbd9d6982 service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.120163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:37.146446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 876/1751] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:37.194273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4202362f8fc4747b9a49a6a6b6f28e6', 'request_id': 'req-669b8e54-484b-4365-bda3-7f11afb3e5aa'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:37.207784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4202362f8fc4747b9a49a6a6b6f28e6', 'request_id': 'req-669b8e54-484b-4365-bda3-7f11afb3e5aa'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:37.228860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4202362f8fc4747b9a49a6a6b6f28e6', 'request_id': 'req-669b8e54-484b-4365-bda3-7f11afb3e5aa'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:37.261339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] The token cache is being invalidate because project b4202362f8fc4747b9a49a6a6b6f28e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:37.261632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:37.262641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-669b8e54-484b-4365-bda3-7f11afb3e5aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:37.266585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 876/1752] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:36 2022] DELETE /identity/v3/projects/b4202362f8fc4747b9a49a6a6b6f28e6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:37.273439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.273904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.274270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.320382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:37.322289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:37.437612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 877/1753] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:37.445523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.445870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.446177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.477383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:37.478798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:37.488310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 877/1754] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:37.492627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.503024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761880c-017c-43e1-a533-c3303aa97839 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.508837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.509602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.509889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.510197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.510922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:37.511255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.513079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.527174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 878/1755] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:01:37.531592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.541833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e48d6-cde5-4df5-a216-e8f5f20b4e2a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.547496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.548129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.548407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.548679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.550017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.552680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.591504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 878/1756] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:37 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:37.596263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.607842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171161dc-805d-4973-b8a8-ff64accff58c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.675378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.676147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.676524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.676875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.678193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.680573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.710520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 879/1757] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:37 2022] POST /identity/v3/users => generated 297 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:37.715125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.727659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71098314-28ed-4587-b175-e64be23b6f6c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.732458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.733067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.733344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.733659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.734369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.736016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.750580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 879/1758] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:37.751613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.762598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf3d93-6071-466d-bb0c-412c97851a88 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.768228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821843a9-e2d8-4aa9-a672-5ee60f9f31cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.768911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821843a9-e2d8-4aa9-a672-5ee60f9f31cd admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.769258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821843a9-e2d8-4aa9-a672-5ee60f9f31cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.769601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821843a9-e2d8-4aa9-a672-5ee60f9f31cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.770355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821843a9-e2d8-4aa9-a672-5ee60f9f31cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.772050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821843a9-e2d8-4aa9-a672-5ee60f9f31cd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.787884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 880/1759] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:37.792581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.803046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99462e-b9e4-43da-ad7b-645365e250c3 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:37.805666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb465f0-69ba-4a22-aff2-ee9391fd4612 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.805956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb465f0-69ba-4a22-aff2-ee9391fd4612 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.806260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb465f0-69ba-4a22-aff2-ee9391fd4612 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.807997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 881/1760] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 24 05:01:37.808802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b1862-9f3a-4fa8-961c-e942b2e9fedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:37.809553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1862-9f3a-4fa8-961c-e942b2e9fedc admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.809927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1862-9f3a-4fa8-961c-e942b2e9fedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.810260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1862-9f3a-4fa8-961c-e942b2e9fedc admin admin] PATH_INFO: `/v3/projects/b46d5bbbd1a741b4a9b5b11033bad56c/users/089c244b8217480ea7a44e1573d2d194/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.826974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b1862-9f3a-4fa8-961c-e942b2e9fedc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46d5bbbd1a741b4a9b5b11033bad56c, user_id=089c244b8217480ea7a44e1573d2d194, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:37.829521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b1862-9f3a-4fa8-961c-e942b2e9fedc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:37.831182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.831681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.831681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.859490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:37.860879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:37.867223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 880/1761] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:37 2022] PUT /identity/v3/projects/b46d5bbbd1a741b4a9b5b11033bad56c/users/089c244b8217480ea7a44e1573d2d194/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:37.869162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:37.869928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:37.869928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:37.900972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:37.911861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] MFA Rules not processed for user `089c244b8217480ea7a44e1573d2d194`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:37.915007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:37.993801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 882/1762] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:37.994694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:37.996007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.035394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 881/1763] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:38.036358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.038872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.060773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe2bcd-7050-4a39-8515-ce59b2673887 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.067047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.067656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.068012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.068205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.076943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.077702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bfa4d-0733-4351-b85a-e7e71cc2a64d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.078633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.082507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.083686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.083686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.083903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] PATH_INFO: `/v3/users/d205927554f64deb87d32a5564d34b82` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.085039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] RBAC: Authorizing `identity:delete_user(user_id=d205927554f64deb87d32a5564d34b82)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.087114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.089591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 882/1764] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:38.090410 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.102209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b50e3af-8a6b-4304-8f76-fbe9a47b8631 service placement] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.107576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.108279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.108629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.109028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.109806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.111872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.125899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:38.150132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 883/1765] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:38 2022] POST /identity/v3/projects => generated 391 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:38.154565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.167447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd205927554f64deb87d32a5564d34b82'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.178652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd205927554f64deb87d32a5564d34b82'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.179370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd205927554f64deb87d32a5564d34b82'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.187188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd205927554f64deb87d32a5564d34b82'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.187910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd205927554f64deb87d32a5564d34b82'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.207180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 883/1766] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:37 2022] DELETE /identity/v3/users/d205927554f64deb87d32a5564d34b82 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:38.227858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b1c6e-38eb-4ab0-a71c-3b56daa87f03 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.292138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.292693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.292967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.293238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.293868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.295851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.325963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 884/1767] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:38 2022] POST /identity/v3/users => generated 302 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:01:38.330131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.331711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.346723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b3d0c-4ece-4d94-8c7c-e6dd8dcdaead admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.352144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.352810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.353163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.353512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.354332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.356188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.368770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 884/1768] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:38.372709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.386250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03312972-50c6-4e82-ac6f-6fad67d3e9bc admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.387595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.389339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.398650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.399438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.399860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.400952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] PATH_INFO: `/v3/projects/1691d72762c8400b97551704077ac195/users/7cbc26cb15f2470f83b868e30e81031e/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.404487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c83a2-72d4-4f50-8f1b-35df6a1cbdba admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.410422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.411181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.411465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.412483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] PATH_INFO: `/v3/projects/92c94dc5b06d4cdbb94682716dd3771f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.413063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c94dc5b06d4cdbb94682716dd3771f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.415252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.441846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1691d72762c8400b97551704077ac195, user_id=7cbc26cb15f2470f83b868e30e81031e, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.443006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:38.446568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.465114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 885/1769] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:38 2022] PUT /identity/v3/projects/1691d72762c8400b97551704077ac195/users/7cbc26cb15f2470f83b868e30e81031e/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:38.468932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.472758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c94dc5b06d4cdbb94682716dd3771f', 'request_id': 'req-d34d5fd3-2893-4045-b90a-b9308a73b33a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.480881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c94dc5b06d4cdbb94682716dd3771f', 'request_id': 'req-d34d5fd3-2893-4045-b90a-b9308a73b33a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.488228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c94dc5b06d4cdbb94682716dd3771f', 'request_id': 'req-d34d5fd3-2893-4045-b90a-b9308a73b33a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.505444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] The token cache is being invalidate because project 92c94dc5b06d4cdbb94682716dd3771f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:38.506572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.506948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.512059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 885/1770] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:38 2022] DELETE /identity/v3/projects/92c94dc5b06d4cdbb94682716dd3771f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:38.513586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.515653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.545119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956b77b-90cb-4de2-b59f-1914c36e4d12 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.550361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.550973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.551248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.551516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.552154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.553895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.560114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:38.567568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 886/1771] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:38.571875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.575136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.590988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed122dc-73db-4cf0-9b22-017ab5c1fcc2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.596689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42a209-53a7-47b9-b057-5aa733306149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.597346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42a209-53a7-47b9-b057-5aa733306149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.597623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42a209-53a7-47b9-b057-5aa733306149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.597895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42a209-53a7-47b9-b057-5aa733306149 admin admin] PATH_INFO: `/v3/projects/1691d72762c8400b97551704077ac195/users/7cbc26cb15f2470f83b868e30e81031e/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.600366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42a209-53a7-47b9-b057-5aa733306149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1691d72762c8400b97551704077ac195, user_id=7cbc26cb15f2470f83b868e30e81031e, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.603197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42a209-53a7-47b9-b057-5aa733306149 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.615558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 887/1772] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:38 2022] PUT /identity/v3/projects/1691d72762c8400b97551704077ac195/users/7cbc26cb15f2470f83b868e30e81031e/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:38.619111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d5fd3-2893-4045-b90a-b9308a73b33a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.626187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.626477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.626761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.657061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] MFA Rules not processed for user `7cbc26cb15f2470f83b868e30e81031e`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:38.658694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.677115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.677683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.677954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.678270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] PATH_INFO: `/v3/users/1dfeea504261439ebbf0f763b732f964` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.679380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dfeea504261439ebbf0f763b732f964)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.681310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.732516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 888/1773] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:38 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:38.734239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dfeea504261439ebbf0f763b732f964'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.743692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.744003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dfeea504261439ebbf0f763b732f964'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.744726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dfeea504261439ebbf0f763b732f964'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.745099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.750718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:38.754527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dfeea504261439ebbf0f763b732f964'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.755802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dfeea504261439ebbf0f763b732f964'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:38.785148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 886/1774] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:38 2022] DELETE /identity/v3/users/1dfeea504261439ebbf0f763b732f964 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:38.804486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b87909-1f40-4644-93c3-036bbb09c637 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:38.861317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:38.862027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:38.862301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:38.862576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:38.864411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:38.908793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:38.931437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:38.932874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:38.943026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 889/1775] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:38 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:38.975491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:38.977223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:39.011782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.013761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:39.027419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d677172-7f07-4010-a52f-e32bd0fbc361 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.044037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.045674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.046962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.047415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.053664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:39.114106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6eec0-e8dc-48ba-aa3b-aef8a63cf737 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.126025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.126686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.126968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.127243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] PATH_INFO: `/v3/projects/320e2bc6838f4f518440111b0f3e31c1` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.129481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=320e2bc6838f4f518440111b0f3e31c1)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.133976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.158090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.160587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c944b4-698d-495f-b186-f77bdf2272a5 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.186302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 890/1776] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:39.225884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '320e2bc6838f4f518440111b0f3e31c1', 'request_id': 'req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:39.241226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '320e2bc6838f4f518440111b0f3e31c1', 'request_id': 'req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:39.250357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '320e2bc6838f4f518440111b0f3e31c1', 'request_id': 'req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:39.269564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] The token cache is being invalidate because project 320e2bc6838f4f518440111b0f3e31c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:39.269911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:39.270820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0b9a6f0e-6bc9-445a-be0d-60f2ecd6e9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:39.284020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 887/1777] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:38 2022] DELETE /identity/v3/projects/320e2bc6838f4f518440111b0f3e31c1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:39.291280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.291640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.291828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.321016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:39.322304 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:39.388614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 891/1778] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:39.396515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.397065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.397263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.423839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:39.425096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:39.438726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 888/1779] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:39.439854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.450654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d470c6-3b84-422f-a0be-04afa6f9e78d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.457129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.457872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.458343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.460415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.461168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:39.461489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.463538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.482361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 892/1780] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:01:39.487816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.501777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdeb7e1-a412-49ea-bcbc-47194e9f56a9 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.507642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.508380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.508763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.509034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.509677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.511623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.556425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 889/1781] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:39 2022] POST /identity/v3/projects => generated 367 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:01:39.560622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.576867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582306e1-a7be-4bb6-8c3c-84e1418a05c1 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.709585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.709832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.710141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.710418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.711046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.712940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.724046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:39.740620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 893/1782] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:39 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:39.744799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.756078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdf953-d33c-470f-b64a-56c675886440 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.760980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.761624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.761904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.762219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.762860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.764459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.777443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 890/1783] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:39.779401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.789703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187be2f-39eb-4ca6-bf80-cfefbacb2a52 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.794417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e2a6d8-a0d3-4416-81ce-f46ceccad5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.795030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a6d8-a0d3-4416-81ce-f46ceccad5cb admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.795307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a6d8-a0d3-4416-81ce-f46ceccad5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.795574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e2a6d8-a0d3-4416-81ce-f46ceccad5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.796743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a6d8-a0d3-4416-81ce-f46ceccad5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.798739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e2a6d8-a0d3-4416-81ce-f46ceccad5cb admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.813584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 894/1784] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:39.817701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:39.829224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ef288-ecb1-4809-9907-e6cab163ce70 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:39.835863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:39.836584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.837002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.837340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] PATH_INFO: `/v3/projects/a6aebe98cbc14e818b977cb9e54e5b70/users/cc952df3884241fbaca2223fdb2e8990/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.853764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6aebe98cbc14e818b977cb9e54e5b70, user_id=cc952df3884241fbaca2223fdb2e8990, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:39.857391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:39.880743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 891/1785] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:39 2022] PUT /identity/v3/projects/a6aebe98cbc14e818b977cb9e54e5b70/users/cc952df3884241fbaca2223fdb2e8990/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:39.894249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:39.894548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:39.894808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:39.931622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] MFA Rules not processed for user `cc952df3884241fbaca2223fdb2e8990`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:39.934175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:40.014302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 895/1786] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:40.026756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:40.028714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:40.034403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:40.114970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec912fc-22ba-416f-87cc-7d80a3968088 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:40.118876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa4e52-0f1b-400f-a1c5-805571001cf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:40.119491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa4e52-0f1b-400f-a1c5-805571001cf9 service glance] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:40.119766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa4e52-0f1b-400f-a1c5-805571001cf9 service glance] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:40.120035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa4e52-0f1b-400f-a1c5-805571001cf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:40.129295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa4e52-0f1b-400f-a1c5-805571001cf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:40.130756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa4e52-0f1b-400f-a1c5-805571001cf9 service glance] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:40.139938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 892/1787] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:42.909053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:42.910558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:42.977405 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2367e27-50b2-4ae1-9d09-d6abb24be8fd None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:42.982101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:42.982735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:42.983018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:42.983285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] PATH_INFO: `/v3/users/089c244b8217480ea7a44e1573d2d194` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:42.984970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=089c244b8217480ea7a44e1573d2d194)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:42.987216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:43.042872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '089c244b8217480ea7a44e1573d2d194'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.052129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '089c244b8217480ea7a44e1573d2d194'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.053233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '089c244b8217480ea7a44e1573d2d194'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.054033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '089c244b8217480ea7a44e1573d2d194'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.070195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '089c244b8217480ea7a44e1573d2d194'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.078802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:43.091608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 896/1788] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:42 2022] DELETE /identity/v3/users/089c244b8217480ea7a44e1573d2d194 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:43.096234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:43.096781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:43.096781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:43.119739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:43.119739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:43.182659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 893/1789] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:43.193308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:43.195299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:43.268277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7d7a9-12cb-4c04-9110-6540487990e7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:43.272058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ed2ce-5414-48df-8908-4df814f71099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:43.272672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ed2ce-5414-48df-8908-4df814f71099 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:43.272949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ed2ce-5414-48df-8908-4df814f71099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:43.273377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ed2ce-5414-48df-8908-4df814f71099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:43.285067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ed2ce-5414-48df-8908-4df814f71099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:43.286817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ed2ce-5414-48df-8908-4df814f71099 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:43.298060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 897/1790] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:43.531576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:43.533075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:43.545436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bf48e-dec8-4e6e-be69-3a105365f920 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:43.550632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:43.551289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:43.551552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:43.551834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] PATH_INFO: `/v3/users/28306a5f694744a2b1812c245eeba4c2` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:43.552958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28306a5f694744a2b1812c245eeba4c2)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:43.555587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:43.613052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28306a5f694744a2b1812c245eeba4c2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.622069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28306a5f694744a2b1812c245eeba4c2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.622772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28306a5f694744a2b1812c245eeba4c2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.631008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28306a5f694744a2b1812c245eeba4c2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.631917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28306a5f694744a2b1812c245eeba4c2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.652981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:43.667572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 894/1791] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:43 2022] DELETE /identity/v3/users/28306a5f694744a2b1812c245eeba4c2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:43.669403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:43.669781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:43.670152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:43.690640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:43.691904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:43.760139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 898/1792] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:43.771090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:43.773494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:43.807525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:43.809610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:43.834325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b09032-41f6-48ed-b0f8-5c2bfc5e2c94 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:43.840654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:43.841455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:43.841757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:43.842041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] PATH_INFO: `/v3/projects/b46d5bbbd1a741b4a9b5b11033bad56c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:43.844164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46d5bbbd1a741b4a9b5b11033bad56c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:43.846559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:43.868990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95091824-294e-40ff-b61d-809efed1deb6 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:43.873099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:43.873933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:43.874120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:43.874438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:43.885354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:43.890303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:43.909085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 895/1793] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:43.920205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46d5bbbd1a741b4a9b5b11033bad56c', 'request_id': 'req-020f165b-fe48-48ba-a669-b6ddfc0271e9'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.928284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46d5bbbd1a741b4a9b5b11033bad56c', 'request_id': 'req-020f165b-fe48-48ba-a669-b6ddfc0271e9'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.936922 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46d5bbbd1a741b4a9b5b11033bad56c', 'request_id': 'req-020f165b-fe48-48ba-a669-b6ddfc0271e9'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.949670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] The token cache is being invalidate because project b46d5bbbd1a741b4a9b5b11033bad56c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:43.950181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.951048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:43.952729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 899/1794] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:43 2022] DELETE /identity/v3/projects/b46d5bbbd1a741b4a9b5b11033bad56c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:43.956710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:43.969246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6bee0-d2f3-4482-82c4-7f5defd926aa service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:44.056213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:44.056989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.057392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.058033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] PATH_INFO: `/v3/users/7cbc26cb15f2470f83b868e30e81031e` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.059166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbc26cb15f2470f83b868e30e81031e)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:44.061164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:44.136023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbc26cb15f2470f83b868e30e81031e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.153421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbc26cb15f2470f83b868e30e81031e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.154272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbc26cb15f2470f83b868e30e81031e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.166455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbc26cb15f2470f83b868e30e81031e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.167156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbc26cb15f2470f83b868e30e81031e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.190045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 896/1795] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:43 2022] DELETE /identity/v3/users/7cbc26cb15f2470f83b868e30e81031e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:44.308566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:44.310257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:44.324066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f165b-fe48-48ba-a669-b6ddfc0271e9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:44.360057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:44.426722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:44.427863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.428148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.428423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] PATH_INFO: `/v3/projects/653aa5e6a8ce469db4b8f20de0d5f5e5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.429611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=653aa5e6a8ce469db4b8f20de0d5f5e5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:44.431565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:44.487908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '653aa5e6a8ce469db4b8f20de0d5f5e5', 'request_id': 'req-764be4b0-8aa2-4598-b075-3f9e6c4cd812'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.498145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '653aa5e6a8ce469db4b8f20de0d5f5e5', 'request_id': 'req-764be4b0-8aa2-4598-b075-3f9e6c4cd812'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.510802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '653aa5e6a8ce469db4b8f20de0d5f5e5', 'request_id': 'req-764be4b0-8aa2-4598-b075-3f9e6c4cd812'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.528779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:44.530345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:44.541350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] The token cache is being invalidate because project 653aa5e6a8ce469db4b8f20de0d5f5e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:44.541350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.541861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.545165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 900/1796] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:44 2022] DELETE /identity/v3/projects/653aa5e6a8ce469db4b8f20de0d5f5e5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:44.549578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:44.552361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895dfd9-2e2b-40a0-b9be-61ba7803c3cf admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:44.553951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:44.590963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:44.642563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:44.643494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.643960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.644556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] PATH_INFO: `/v3/projects/1691d72762c8400b97551704077ac195` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.646586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1691d72762c8400b97551704077ac195)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:44.649137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:44.649272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764be4b0-8aa2-4598-b075-3f9e6c4cd812 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:44.661210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:44.663056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.663293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.663549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] PATH_INFO: `/v3/users/70a2dfacd3dd47c39343ab0050c0147b` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.666786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a2dfacd3dd47c39343ab0050c0147b)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:44.674506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:44.726679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1691d72762c8400b97551704077ac195', 'request_id': 'req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.741882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1691d72762c8400b97551704077ac195', 'request_id': 'req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.749739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1691d72762c8400b97551704077ac195', 'request_id': 'req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.759117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a2dfacd3dd47c39343ab0050c0147b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.763640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:44.768256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] The token cache is being invalidate because project 1691d72762c8400b97551704077ac195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:44.774286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.775278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a2dfacd3dd47c39343ab0050c0147b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.776160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a2dfacd3dd47c39343ab0050c0147b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.778226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f684e52d-28ba-4520-9bc8-bbd90e2d1c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.778359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a2dfacd3dd47c39343ab0050c0147b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.782753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 897/1797] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:44 2022] DELETE /identity/v3/projects/1691d72762c8400b97551704077ac195 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:44.785645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c46ad697-ac9a-45a8-ac35-232cc0da2b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a2dfacd3dd47c39343ab0050c0147b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:44.787726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.788090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.788434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.812254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:44.814072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:44.815897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 901/1798] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:44 2022] DELETE /identity/v3/users/70a2dfacd3dd47c39343ab0050c0147b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:44.887499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 898/1799] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:44.895776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.896316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.896595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.927628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:44.928772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:44.943616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 902/1800] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:44.948021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:44.958652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbf656-900e-457f-bbd7-045d2563a8a0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:44.963569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:44.964182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:44.964526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:44.964719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:44.965364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:44.965706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:44.967339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:44.980703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 899/1801] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:01:44.985534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:44.995284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9d1f2-d789-4adc-85da-08e70d6984c9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.005855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.006398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.006724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.007082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.007801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.009889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.056892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 903/1802] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:44 2022] POST /identity/v3/projects => generated 409 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:45.058138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.070621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26240d-ccbd-42a8-8c42-60d231f874a6 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.110397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:45.147512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.150276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.150276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.150276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.150276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.152312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.182051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 900/1803] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/users => generated 305 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:45.185058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.197153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01975be-c540-4d60-a5e8-5132a84d5486 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.203213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.203840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.204119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.204393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.205040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.206878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.217300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 904/1804] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:45.221676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.231878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c40c52-55eb-4621-86f7-0914a13d8f9f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.237657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6b705-2ca0-432a-90e1-2390f211ec43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.238310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6b705-2ca0-432a-90e1-2390f211ec43 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.238591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6b705-2ca0-432a-90e1-2390f211ec43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.238864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6b705-2ca0-432a-90e1-2390f211ec43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.239504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6b705-2ca0-432a-90e1-2390f211ec43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.241092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6b705-2ca0-432a-90e1-2390f211ec43 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.252905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 901/1805] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:45.256916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.270839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145c756-2ccd-4f26-890e-e5a00fdfd93b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.276372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.276794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.277071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.277461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] PATH_INFO: `/v3/projects/69ff3919676a4299800affa95cdeed4f/users/1fee7795dcaf4fcf92d9324f3ebbdad3/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.295103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69ff3919676a4299800affa95cdeed4f, user_id=1fee7795dcaf4fcf92d9324f3ebbdad3, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.299121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.313264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 905/1806] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:45 2022] PUT /identity/v3/projects/69ff3919676a4299800affa95cdeed4f/users/1fee7795dcaf4fcf92d9324f3ebbdad3/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:45.324903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.325261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.325658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.332864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.334858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:45.359484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] MFA Rules not processed for user `1fee7795dcaf4fcf92d9324f3ebbdad3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:45.361247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:45.466422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4246ab2-9adc-420e-9032-33112416a870 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.472953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.473596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.473881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.474182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] PATH_INFO: `/v3/projects/7b4c234ef410476f85a29dbde06b47a6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.475497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4c234ef410476f85a29dbde06b47a6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.477469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 902/1807] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:45.478722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.481619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.508856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397a13c-ce05-4667-9833-5969aa081318 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.525411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.526444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.527028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.527490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.528813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.531362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.576730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 903/1808] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/projects => generated 407 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:45.580776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4c234ef410476f85a29dbde06b47a6', 'request_id': 'req-09cca69c-c564-440a-a831-e1e47c7c6c71'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:45.581540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.588084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4c234ef410476f85a29dbde06b47a6', 'request_id': 'req-09cca69c-c564-440a-a831-e1e47c7c6c71'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:45.602043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4c234ef410476f85a29dbde06b47a6', 'request_id': 'req-09cca69c-c564-440a-a831-e1e47c7c6c71'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:45.605752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:45.628724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] The token cache is being invalidate because project 7b4c234ef410476f85a29dbde06b47a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:45.629128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:45.629836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09cca69c-c564-440a-a831-e1e47c7c6c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:45.632126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 906/1809] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:45 2022] DELETE /identity/v3/projects/7b4c234ef410476f85a29dbde06b47a6 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:01:45.638586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272eb69a-2a48-4179-9c12-fe1bf032a2cf None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.638985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272eb69a-2a48-4179-9c12-fe1bf032a2cf None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.639270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272eb69a-2a48-4179-9c12-fe1bf032a2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.668756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-272eb69a-2a48-4179-9c12-fe1bf032a2cf None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:45.670487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-272eb69a-2a48-4179-9c12-fe1bf032a2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:45.694012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ca323-f7bb-4050-ad61-ce88824fed3e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.698824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.699723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.699845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.700059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.700698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.703274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.730873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 904/1810] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/users => generated 310 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:45.735607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.741078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:45.746290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 907/1811] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:45.754981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93e40-3205-4528-aac4-36beed049985 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.755194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.755443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.755911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.759981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.760608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.760878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.761170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.761807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.763439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.778351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 905/1812] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:45.778563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:45.779228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.779792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:45.784321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:45.793292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 908/1813] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:45.793487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d93cf7-b186-4543-a525-67e620427f1e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.794918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.799027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.800086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.800455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.800826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] PATH_INFO: `/v3/projects/dd257ffed4794e229dbf1475f0c220ac/users/9a69d672f63640f69197d942c037eaa0/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.816744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133534d5-0b27-43d7-9f78-2c97e8007c51 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.823703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd257ffed4794e229dbf1475f0c220ac, user_id=9a69d672f63640f69197d942c037eaa0, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.824566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.825897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.826204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.826480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.827662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:45.827662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.827960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.829051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.848369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 909/1814] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:01:45.850419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.856423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 906/1815] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:45 2022] PUT /identity/v3/projects/dd257ffed4794e229dbf1475f0c220ac/users/9a69d672f63640f69197d942c037eaa0/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:45.857422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:45.861341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52128d8f-707b-4ea1-9117-5f8f2a29272f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.949763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515ccb96-ec55-4743-acdb-6e07dd746fe2 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:45.965205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.966154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.966523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.966807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.967625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.969372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:45.969893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.972524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:45.972524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:45.972910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:45.974775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:45.978661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:45.981415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 907/1816] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:45.988906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.004822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2adcf-0861-4f7a-8a0d-58b633559e15 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.012818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fee7c5-d413-49f2-9337-aba97a10467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.014250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fee7c5-d413-49f2-9337-aba97a10467c admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.016280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fee7c5-d413-49f2-9337-aba97a10467c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.016652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fee7c5-d413-49f2-9337-aba97a10467c admin admin] PATH_INFO: `/v3/projects/dd257ffed4794e229dbf1475f0c220ac/users/9a69d672f63640f69197d942c037eaa0/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.021208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fee7c5-d413-49f2-9337-aba97a10467c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd257ffed4794e229dbf1475f0c220ac, user_id=9a69d672f63640f69197d942c037eaa0, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.026144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fee7c5-d413-49f2-9337-aba97a10467c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.030526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 910/1817] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:45 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:01:46.041504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.060540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 908/1818] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:45 2022] PUT /identity/v3/projects/dd257ffed4794e229dbf1475f0c220ac/users/9a69d672f63640f69197d942c037eaa0/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:46.069552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.069701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.069894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.123369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] MFA Rules not processed for user `9a69d672f63640f69197d942c037eaa0`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:46.126388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:46.210548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9b223-cf89-4159-a54a-cce2f71e873b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.216857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:46.275905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 909/1819] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:46.285571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.287058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:46.304651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.305384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.305672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.305952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.306675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.309287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.345910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 911/1820] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:46 2022] POST /identity/v3/users => generated 284 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:46.351014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.372499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e0293-1493-457b-8881-828d1febf1a8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.378626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:46.383228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.384353 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.384744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.385259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.386153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.388744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.394615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4c06f-e2df-4e22-bf09-0dfb447ee4c1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.402757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e49d69-5ce6-4e91-8bfb-6b4ea486487b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.403733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49d69-5ce6-4e91-8bfb-6b4ea486487b service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.404156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49d69-5ce6-4e91-8bfb-6b4ea486487b service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.404739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49d69-5ce6-4e91-8bfb-6b4ea486487b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.406740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 912/1821] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:46.410610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.423908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59f076-119e-47fa-83bc-f867b299fb4d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.429211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.429836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.430139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.430436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.434326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.437472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.448841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e49d69-5ce6-4e91-8bfb-6b4ea486487b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.452754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e49d69-5ce6-4e91-8bfb-6b4ea486487b service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.481444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 913/1822] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:46.482418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.491249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 910/1823] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:46 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:46.505080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f015b3-c549-4cd5-b480-a4093fce2763 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.515896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.516909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.517703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.517703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] PATH_INFO: `/v3/projects/def435aede05402bb6cc8175a3f9d276/users/cf9c447ef097496cbd63f57bb11b08cd/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.550493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def435aede05402bb6cc8175a3f9d276, user_id=cf9c447ef097496cbd63f57bb11b08cd, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.554430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.601431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 914/1824] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:46 2022] PUT /identity/v3/projects/def435aede05402bb6cc8175a3f9d276/users/cf9c447ef097496cbd63f57bb11b08cd/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:46.605328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.605725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.606040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.657396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] MFA Rules not processed for user `cf9c447ef097496cbd63f57bb11b08cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:46.661052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:46.741630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.759295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 911/1825] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:46.763915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.821940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a57fe-a7ab-4c1f-8ad3-21fbccf45d73 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.826889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.827525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.827991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.828339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.830513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:46.841758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2539f-68ff-4332-879e-96ed5230ad14 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:46.847062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:46.847696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:46.847972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:46.848229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:46.849482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.851467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.905719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 912/1826] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:46 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:01:46.909174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:46.918767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:46.920478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:46.932279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 915/1827] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:46 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:01:47.001291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba574a-a8f1-422c-865e-f8106e541bb8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:47.006178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:47.006755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.007031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.007304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.007934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:47.009856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:47.020786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:47.034912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 913/1828] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:46 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:01:47.038879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:47.045875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:47.050748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dedec-3287-417d-937e-9877d561076b service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:47.056681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:47.057363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.057645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.057922 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.058594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:47.060212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:47.078683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 916/1829] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:47.084235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:47.096172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee04777-bf9e-4127-9b92-10714713c3c8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:47.101538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:47.102439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.102838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.103441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] PATH_INFO: `/v3/projects/301f9d71c9fe4640ac43c293cf17a8bf/users/16983380e67e46ffaa88fcbba4ff16b6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.126694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=301f9d71c9fe4640ac43c293cf17a8bf, user_id=16983380e67e46ffaa88fcbba4ff16b6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:47.130070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:47.167017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 914/1830] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:47 2022] PUT /identity/v3/projects/301f9d71c9fe4640ac43c293cf17a8bf/users/16983380e67e46ffaa88fcbba4ff16b6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:47.170706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:47.184844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f8f34d-e867-4427-9658-906f5dc09c83 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:47.286907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c449bfc-3e91-4081-9259-9c6ddac3265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:47.287453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c449bfc-3e91-4081-9259-9c6ddac3265c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.287748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c449bfc-3e91-4081-9259-9c6ddac3265c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.288025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c449bfc-3e91-4081-9259-9c6ddac3265c admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.288679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c449bfc-3e91-4081-9259-9c6ddac3265c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:47.292146 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c449bfc-3e91-4081-9259-9c6ddac3265c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:47.324902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 917/1831] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:47.329275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:47.341571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f454dac-b88e-4612-9d69-9ccf7f677c89 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:47.349990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:47.351148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.351610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.352011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] PATH_INFO: `/v3/projects/301f9d71c9fe4640ac43c293cf17a8bf/users/16983380e67e46ffaa88fcbba4ff16b6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.354556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] RBAC: Authorizing `identity:create_grant(project_id=301f9d71c9fe4640ac43c293cf17a8bf, user_id=16983380e67e46ffaa88fcbba4ff16b6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:47.358067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:47.378866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 915/1832] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:47 2022] PUT /identity/v3/projects/301f9d71c9fe4640ac43c293cf17a8bf/users/16983380e67e46ffaa88fcbba4ff16b6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:47.391181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217a729-41a2-443d-9391-99380cc86957 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.392067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217a729-41a2-443d-9391-99380cc86957 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.392524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217a729-41a2-443d-9391-99380cc86957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.430734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0217a729-41a2-443d-9391-99380cc86957 None None] MFA Rules not processed for user `16983380e67e46ffaa88fcbba4ff16b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:47.432174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0217a729-41a2-443d-9391-99380cc86957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:47.501734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 918/1833] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:47.512867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:47.569273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d264d-cac6-460f-8301-e6c175b5046f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:47.572940 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92145c2d-c5fc-4612-ad7b-b01445d59d3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:47.573568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145c2d-c5fc-4612-ad7b-b01445d59d3a service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:47.573859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145c2d-c5fc-4612-ad7b-b01445d59d3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:47.574177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92145c2d-c5fc-4612-ad7b-b01445d59d3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:47.584392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92145c2d-c5fc-4612-ad7b-b01445d59d3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:47.585821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92145c2d-c5fc-4612-ad7b-b01445d59d3a service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:47.596648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 916/1834] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:49.131455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217a729-41a2-443d-9391-99380cc86957 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:49.197016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217a729-41a2-443d-9391-99380cc86957 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:49.201478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:49.202132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:49.203044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:49.203339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] PATH_INFO: `/v3/users/cf9c447ef097496cbd63f57bb11b08cd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:49.204421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf9c447ef097496cbd63f57bb11b08cd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:49.206373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:49.221686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:49.251473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf9c447ef097496cbd63f57bb11b08cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.258849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf9c447ef097496cbd63f57bb11b08cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.259579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf9c447ef097496cbd63f57bb11b08cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.260251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf9c447ef097496cbd63f57bb11b08cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.267968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf9c447ef097496cbd63f57bb11b08cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.284657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 919/1835] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:49 2022] DELETE /identity/v3/users/cf9c447ef097496cbd63f57bb11b08cd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:49.289074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:49.289539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:49.289628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:49.307887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:49.309029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:49.368390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 917/1836] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:49.378775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:49.380208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:49.436535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0741058-ca72-4815-b7a7-5e2af1d1081f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:49.440586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:49.441204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:49.441480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:49.441746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:49.450552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:49.451955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:49.461826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 920/1837] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:01:49.743746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:49.745232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:49.758123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1844-932a-4970-b467-4b20702c3d85 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:49.763225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:49.763924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:49.764204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:49.764473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] PATH_INFO: `/v3/projects/def435aede05402bb6cc8175a3f9d276` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:49.766483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=def435aede05402bb6cc8175a3f9d276)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:49.768385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:49.846831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def435aede05402bb6cc8175a3f9d276', 'request_id': 'req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.850424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:49.854566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def435aede05402bb6cc8175a3f9d276', 'request_id': 'req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.862731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def435aede05402bb6cc8175a3f9d276', 'request_id': 'req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.876333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] The token cache is being invalidate because project def435aede05402bb6cc8175a3f9d276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:49.876746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.878856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:49.880881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 918/1838] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:49 2022] DELETE /identity/v3/projects/def435aede05402bb6cc8175a3f9d276 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:49.884728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:49.895351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478b9ec-2c56-46ff-a1a1-4e9ae7f36a14 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:49.959889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:49.960562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:49.960836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:49.961106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] PATH_INFO: `/v3/users/16983380e67e46ffaa88fcbba4ff16b6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:49.962287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=16983380e67e46ffaa88fcbba4ff16b6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:49.964579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.012490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:50.031122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16983380e67e46ffaa88fcbba4ff16b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.038249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16983380e67e46ffaa88fcbba4ff16b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.038867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16983380e67e46ffaa88fcbba4ff16b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.039475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16983380e67e46ffaa88fcbba4ff16b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.046667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d853cca4-4fef-428a-b569-fd6bc38f5bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16983380e67e46ffaa88fcbba4ff16b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.064714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 921/1839] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:01:49 2022] DELETE /identity/v3/users/16983380e67e46ffaa88fcbba4ff16b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:50.368073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.369518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:50.385408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9258e1-ad9e-4dd0-9309-b8fbb4635bc7 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.452873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.453512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.453791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.454096 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] PATH_INFO: `/v3/projects/301f9d71c9fe4640ac43c293cf17a8bf` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.455218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=301f9d71c9fe4640ac43c293cf17a8bf)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.457124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.505866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301f9d71c9fe4640ac43c293cf17a8bf', 'request_id': 'req-dcd1cfe7-914a-4586-afd9-8dfef70fd048'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.513061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301f9d71c9fe4640ac43c293cf17a8bf', 'request_id': 'req-dcd1cfe7-914a-4586-afd9-8dfef70fd048'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.518947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301f9d71c9fe4640ac43c293cf17a8bf', 'request_id': 'req-dcd1cfe7-914a-4586-afd9-8dfef70fd048'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.530241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] The token cache is being invalidate because project 301f9d71c9fe4640ac43c293cf17a8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:01:50.530550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.531292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-dcd1cfe7-914a-4586-afd9-8dfef70fd048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:01:50.532812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 919/1840] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:01:50 2022] DELETE /identity/v3/projects/301f9d71c9fe4640ac43c293cf17a8bf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:50.537704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.538217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.538306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.558509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:50.559693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:50.624577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 922/1841] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:01:50.636949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.637166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.637445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.658400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:50.659609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:50.667881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 920/1842] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:50.672758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.682626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cbe10-427c-4421-b170-15b57a8d1f55 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.687292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.687948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.688222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.688500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.689152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:01:50.689450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.691087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.704570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 923/1843] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:01:50.708935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.718647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d491b54-2af1-46d8-995b-372a6e79e3fb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.723267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.723865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.724144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.724416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.725056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.727003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.757453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 921/1844] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:50 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:50.761755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.772276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e184ace-8565-4157-8260-9086318d055d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.836807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.837686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.838181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.838674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.839503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.841444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.868537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 924/1845] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:50 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:01:50.869783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.880648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540324e-376b-40d9-868a-82214f5cdf40 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.885537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.886179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.886453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.886722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.887412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.888934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.899724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 922/1846] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:50.903697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.914312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc913f00-5d8d-46fd-930f-39f8b9a638b6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.918854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef18a2e-1673-4350-87d7-811fd02fa2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.919452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef18a2e-1673-4350-87d7-811fd02fa2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.919731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef18a2e-1673-4350-87d7-811fd02fa2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.920001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef18a2e-1673-4350-87d7-811fd02fa2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.920637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef18a2e-1673-4350-87d7-811fd02fa2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.922372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef18a2e-1673-4350-87d7-811fd02fa2e6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.933138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 925/1847] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:50.938196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:50.948260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc72926e-6429-4708-b3dc-6cc8412b610e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:50.953422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:50.954114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:50.954393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:50.954665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] PATH_INFO: `/v3/projects/c98c34167630486fba99bbb0f10dfe0a/users/8bd7eb1f2594444885b19052ae53ec0d/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:50.970724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c98c34167630486fba99bbb0f10dfe0a, user_id=8bd7eb1f2594444885b19052ae53ec0d, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:50.973452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:50.988073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 923/1848] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:50 2022] PUT /identity/v3/projects/c98c34167630486fba99bbb0f10dfe0a/users/8bd7eb1f2594444885b19052ae53ec0d/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:51.000170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.000447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.000803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.028706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] MFA Rules not processed for user `8bd7eb1f2594444885b19052ae53ec0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:51.029908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:51.038982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:51.099130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 926/1849] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:51.103900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.115961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:51.172212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f8c14-b8be-410b-823a-6ad41e8af0d3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.176840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.177450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.177728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.178017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.178701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.180695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.208863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 924/1850] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:01:51 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:51.213087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.224774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c4e02-a737-470b-bdd6-05b7ed580f7d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.282412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.283029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.283342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.283617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.284290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.286371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.308142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 927/1851] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:01:51 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:01:51.313037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.323907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14498d-6704-4610-9f75-be109a20bad9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.329396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.330125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.330450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.330800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.331611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.333247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.344145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 925/1852] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:01:51.348283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.358703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb1b67-dd82-40e2-97b4-99d1ce1fc0e1 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.363784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.364442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.364747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.365195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] PATH_INFO: `/v3/projects/257dde259e7f4b179b53ce0dea2c87f0/users/9d09e59f40cd4433aa0e9e70f178a50f/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.381768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] RBAC: Authorizing `identity:create_grant(project_id=257dde259e7f4b179b53ce0dea2c87f0, user_id=9d09e59f40cd4433aa0e9e70f178a50f, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.384537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.398485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 928/1853] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:51 2022] PUT /identity/v3/projects/257dde259e7f4b179b53ce0dea2c87f0/users/9d09e59f40cd4433aa0e9e70f178a50f/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:51.402913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.413382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14056ffc-9363-43cc-81aa-db80bb9c464d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.474755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72171bc6-2b82-4314-853a-a7844ee1fd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.475460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72171bc6-2b82-4314-853a-a7844ee1fd16 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.475817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72171bc6-2b82-4314-853a-a7844ee1fd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.476148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72171bc6-2b82-4314-853a-a7844ee1fd16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.476939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72171bc6-2b82-4314-853a-a7844ee1fd16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.478764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72171bc6-2b82-4314-853a-a7844ee1fd16 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.494452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 926/1854] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:01:51.506164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.516669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1e7c2-6f24-454a-8dbb-4cb26025454c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.521340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.521978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.522286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.522557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] PATH_INFO: `/v3/projects/257dde259e7f4b179b53ce0dea2c87f0/users/9d09e59f40cd4433aa0e9e70f178a50f/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.524466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257dde259e7f4b179b53ce0dea2c87f0, user_id=9d09e59f40cd4433aa0e9e70f178a50f, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.527142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.538153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 929/1855] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:01:51 2022] PUT /identity/v3/projects/257dde259e7f4b179b53ce0dea2c87f0/users/9d09e59f40cd4433aa0e9e70f178a50f/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:01:51.548767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.549050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.549327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.576762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] MFA Rules not processed for user `9d09e59f40cd4433aa0e9e70f178a50f`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:01:51.577990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:51.644922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 927/1856] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:01:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:01:51.656053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:01:51.657787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:01:51.714380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a2ee-e5fa-4577-b5f4-1f393b3569f5 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:01:51.718599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:01:51.719214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:01:51.719489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:01:51.719759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:01:51.735904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:01:51.775300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:01:51.776724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:01:51.786426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 930/1857] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:01:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:01.642116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:01.782203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=19 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:01.869605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:01.871061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:01.883221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d63c7-c66c-4d16-8beb-bc46d0a1fb2b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:01.886779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f1b1e-c0d7-4507-8038-5f06aafce725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:01.887394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1b1e-c0d7-4507-8038-5f06aafce725 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:01.887666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1b1e-c0d7-4507-8038-5f06aafce725 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:01.887933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f1b1e-c0d7-4507-8038-5f06aafce725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:01.897539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f1b1e-c0d7-4507-8038-5f06aafce725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:01.898928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f1b1e-c0d7-4507-8038-5f06aafce725 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:01.909019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 928/1858] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:03.372912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:03.374359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:03.440182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf941b28-2988-469d-aabc-c53988ad13de service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:03.444646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:03.445274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:03.445545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:03.445822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] PATH_INFO: `/v3/users/1fee7795dcaf4fcf92d9324f3ebbdad3` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:03.446951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fee7795dcaf4fcf92d9324f3ebbdad3)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:03.448840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:03.492314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fee7795dcaf4fcf92d9324f3ebbdad3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:03.499747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fee7795dcaf4fcf92d9324f3ebbdad3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:03.500466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fee7795dcaf4fcf92d9324f3ebbdad3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:03.501078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fee7795dcaf4fcf92d9324f3ebbdad3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:03.507878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fee7795dcaf4fcf92d9324f3ebbdad3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:03.525730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 931/1859] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:03 2022] DELETE /identity/v3/users/1fee7795dcaf4fcf92d9324f3ebbdad3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:02:03.530297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:03.530788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:03.530883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:03.549173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:03.550405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:03.608650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 929/1860] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:02:03.618228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:03.619641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:03.681193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf0a42-ba37-4659-bede-69654c295f85 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:03.685030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:03.685669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:03.685946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:03.686252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:03.695485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:03.696908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:03.706788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 932/1861] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:03.972692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:03.974452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:03.993396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14144d0-e9ca-46f8-b54a-c2e0e0dab3b4 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:03.998995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:04.000321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.000602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.000879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] PATH_INFO: `/v3/projects/69ff3919676a4299800affa95cdeed4f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.008802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69ff3919676a4299800affa95cdeed4f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:04.022222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:04.059310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69ff3919676a4299800affa95cdeed4f', 'request_id': 'req-da18cdc3-5f55-4822-ab10-f6e47a499c91'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.067044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69ff3919676a4299800affa95cdeed4f', 'request_id': 'req-da18cdc3-5f55-4822-ab10-f6e47a499c91'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.072720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69ff3919676a4299800affa95cdeed4f', 'request_id': 'req-da18cdc3-5f55-4822-ab10-f6e47a499c91'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.084415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] The token cache is being invalidate because project 69ff3919676a4299800affa95cdeed4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:04.084712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.085357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.086888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 930/1862] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:03 2022] DELETE /identity/v3/projects/69ff3919676a4299800affa95cdeed4f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:04.090927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:04.110942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809941d3-beea-4c83-bfc2-b941c47e7d4c service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:04.132908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:04.194064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:04.194064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.194064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.194064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] PATH_INFO: `/v3/users/9a69d672f63640f69197d942c037eaa0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.194525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a69d672f63640f69197d942c037eaa0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:04.194525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:04.248344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a69d672f63640f69197d942c037eaa0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.257139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a69d672f63640f69197d942c037eaa0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.257868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a69d672f63640f69197d942c037eaa0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.258624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a69d672f63640f69197d942c037eaa0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.266523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0415b47b-38a0-4b53-90aa-d5907f76ebcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a69d672f63640f69197d942c037eaa0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.284720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 933/1863] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:04 2022] DELETE /identity/v3/users/9a69d672f63640f69197d942c037eaa0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:04.587765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:04.589421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:04.603783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18cdc3-5f55-4822-ab10-f6e47a499c91 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:04.672319 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:04.672970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.673252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.673525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] PATH_INFO: `/v3/projects/dd257ffed4794e229dbf1475f0c220ac` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.674704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd257ffed4794e229dbf1475f0c220ac)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:04.676740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:04.689653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:04.727993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd257ffed4794e229dbf1475f0c220ac', 'request_id': 'req-6795734c-e765-4ae6-a9c5-3745338c0564'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.735887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd257ffed4794e229dbf1475f0c220ac', 'request_id': 'req-6795734c-e765-4ae6-a9c5-3745338c0564'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.741451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd257ffed4794e229dbf1475f0c220ac', 'request_id': 'req-6795734c-e765-4ae6-a9c5-3745338c0564'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.753154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] The token cache is being invalidate because project dd257ffed4794e229dbf1475f0c220ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:04.753452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.754143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6795734c-e765-4ae6-a9c5-3745338c0564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:04.755911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 931/1864] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:04 2022] DELETE /identity/v3/projects/dd257ffed4794e229dbf1475f0c220ac => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:04.760751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.761219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.761339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.783291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:04.784491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:04.842315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 934/1865] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:04.851743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.852034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.852312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.872310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:04.873503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:04.881789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 932/1866] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:04.886091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:04.896004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622f6a3-45ac-42eb-b990-e9ef1e475c07 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:04.900511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:04.901100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.901379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.901652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.902328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:02:04.902631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:04.904292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:04.916487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 935/1867] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:02:04.921027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:04.930202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f3166-cefe-48bc-8dcf-be8208bb1ab9 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:04.934788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:04.935380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:04.935660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:04.935935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:04.937089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:04.939032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:04.965905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 933/1868] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:04 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:04.969978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:04.979794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad159-99d3-4203-b7ba-ebe7bd1097bf admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.005162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:05.038073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.038685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.038960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.039225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.039850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.041765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.065091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 936/1869] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:04 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:05.069189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.080090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f5b2f-5113-4ab7-88ae-151f6a37ff42 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.085382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.085978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.086284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.086553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.087183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.088757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.100392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 934/1870] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:05.104321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.116951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e2ef4d-9aa8-4e09-9059-9ee6da55c1c9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.121516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19db59-141b-4481-81c3-95b1c91f4f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.122140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19db59-141b-4481-81c3-95b1c91f4f7c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.122414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19db59-141b-4481-81c3-95b1c91f4f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.122702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19db59-141b-4481-81c3-95b1c91f4f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.123352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19db59-141b-4481-81c3-95b1c91f4f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.125450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19db59-141b-4481-81c3-95b1c91f4f7c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.136758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 937/1871] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:02:05.141817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.151533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e53694-70a2-419e-86fd-d013ae72e068 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.156136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.156723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.156999 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.157271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] PATH_INFO: `/v3/projects/a863597cbf7c4fdabe033880aac0995b/users/8b6c5ab418cc406da83c6205382dd1ce/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.172223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a863597cbf7c4fdabe033880aac0995b, user_id=8b6c5ab418cc406da83c6205382dd1ce, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.174954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.188673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 935/1872] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:05 2022] PUT /identity/v3/projects/a863597cbf7c4fdabe033880aac0995b/users/8b6c5ab418cc406da83c6205382dd1ce/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:05.200796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.201079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.201351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.229468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] MFA Rules not processed for user `8b6c5ab418cc406da83c6205382dd1ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:05.230771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:05.301223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 938/1873] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:05.305775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.367605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb759a57-483f-4a73-8c11-3deeccbb733f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.372511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.373199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.373481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.373750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.375107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.377006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.409310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 936/1874] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:05 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:05.410379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.421759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ae2b3-84f4-45f8-8ff3-eb07730d7be1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.486988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.487610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.488029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.488581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.489210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.491822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.517217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 939/1875] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:05 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:02:05.518499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.528923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd632a68-389c-45b7-b09e-226335f388b3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.533433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.534064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.534354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.534627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.535260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.536860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.550901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 937/1876] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:05.552449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.570442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a582b9-97fd-4a74-afe4-ba49587acd07 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.577808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54682338-422b-4265-ad57-00091dd437f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.578439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682338-422b-4265-ad57-00091dd437f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.578725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682338-422b-4265-ad57-00091dd437f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.579002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54682338-422b-4265-ad57-00091dd437f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.579668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54682338-422b-4265-ad57-00091dd437f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.581337 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54682338-422b-4265-ad57-00091dd437f3 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.591904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 940/1877] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:05.596594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.606569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c075b6c-08e2-4f1b-8c2c-1760f459e8f8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.611762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.612455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.612727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.612998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] PATH_INFO: `/v3/projects/a056e7c38bea426bbbad7023d2ab9552/users/84066a219aec435580240e023e224dde/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.621170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:05.628691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a056e7c38bea426bbbad7023d2ab9552, user_id=84066a219aec435580240e023e224dde, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.632180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.645306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 938/1878] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:05 2022] PUT /identity/v3/projects/a056e7c38bea426bbbad7023d2ab9552/users/84066a219aec435580240e023e224dde/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:05.656036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d7551-ca03-4495-93db-82576d737666 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.656319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d7551-ca03-4495-93db-82576d737666 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.656590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d7551-ca03-4495-93db-82576d737666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.684514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-be3d7551-ca03-4495-93db-82576d737666 None None] MFA Rules not processed for user `84066a219aec435580240e023e224dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:05.685693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-be3d7551-ca03-4495-93db-82576d737666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:05.690585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:05.749902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 941/1879] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:05.754428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.818575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a18f3-231a-4c12-a448-b4ac82fb3cb9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.823249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.823858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.824134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.824403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.825035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.826976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.854560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 939/1880] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:05 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:05.859222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d7551-ca03-4495-93db-82576d737666 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.871300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d7551-ca03-4495-93db-82576d737666 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.931429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.932116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.932398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.932670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.933365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.935529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.960789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 942/1881] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:05 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:02:05.961952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:05.972225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc919c1-fc50-4dd2-b1c3-d35b0b8bf9e3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:05.976860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:05.977465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:05.977770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:05.978076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:05.978725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:05.980323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:05.995766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 940/1882] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:05.998675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:06.011525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420bf36-3fd0-41fd-b6ed-e0d2a6965b9a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:06.017462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:06.018283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:06.018657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:06.019029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] PATH_INFO: `/v3/projects/4249abebce2d49cb85e01209c2001a3e/users/303df38f2276453d87f9fe752a868c64/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:06.038733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4249abebce2d49cb85e01209c2001a3e, user_id=303df38f2276453d87f9fe752a868c64, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:06.041419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:06.054404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 943/1883] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:05 2022] PUT /identity/v3/projects/4249abebce2d49cb85e01209c2001a3e/users/303df38f2276453d87f9fe752a868c64/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:06.058332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:06.068182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda17dc6-5d44-40d7-8f49-a0f5cd529088 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:06.127443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b34f4-b757-4816-88b5-969c548daa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:06.128095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b34f4-b757-4816-88b5-969c548daa41 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:06.128401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b34f4-b757-4816-88b5-969c548daa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:06.128673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b34f4-b757-4816-88b5-969c548daa41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:06.129449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b34f4-b757-4816-88b5-969c548daa41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:06.131046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b34f4-b757-4816-88b5-969c548daa41 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:06.142175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 941/1884] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:06.146154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:06.157125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f6e4c-f46a-4bb8-a6f2-551c1cd580de admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:06.161917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:06.162606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:06.162908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:06.163235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] PATH_INFO: `/v3/projects/4249abebce2d49cb85e01209c2001a3e/users/303df38f2276453d87f9fe752a868c64/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:06.165202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4249abebce2d49cb85e01209c2001a3e, user_id=303df38f2276453d87f9fe752a868c64, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:06.167971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:06.178550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 944/1885] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:06 2022] PUT /identity/v3/projects/4249abebce2d49cb85e01209c2001a3e/users/303df38f2276453d87f9fe752a868c64/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:06.189337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:06.189567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:06.189847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:06.218662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] MFA Rules not processed for user `303df38f2276453d87f9fe752a868c64`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:06.219914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:06.262720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:06.288938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 942/1886] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:06.309197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:06.310968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:06.371056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56640d-4308-4122-887f-d71df1fecbd0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:06.376626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:06.377729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:06.378212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:06.378491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:06.390777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:06.392365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:06.397702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:06.403980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 945/1887] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:06.444342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:06.454331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488e7d4-01e5-4d0d-90e3-2e9c0e98288a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:06.458209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf94d75-07bc-447c-9433-45c8a2a866ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:06.458870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf94d75-07bc-447c-9433-45c8a2a866ff service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:06.459142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf94d75-07bc-447c-9433-45c8a2a866ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:06.459418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf94d75-07bc-447c-9433-45c8a2a866ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:06.520031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf94d75-07bc-447c-9433-45c8a2a866ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:06.521460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf94d75-07bc-447c-9433-45c8a2a866ff service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:06.530195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 943/1888] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:16.526335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=36 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:17.948207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:18.013056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944cce3-7bb6-4eb9-84ee-002b70d489ba service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:18.018743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:18.019393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:18.019669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:18.019952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] PATH_INFO: `/v3/users/8b6c5ab418cc406da83c6205382dd1ce` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:18.021079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6c5ab418cc406da83c6205382dd1ce)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:18.023067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:18.074850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6c5ab418cc406da83c6205382dd1ce'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.084600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6c5ab418cc406da83c6205382dd1ce'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.085314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6c5ab418cc406da83c6205382dd1ce'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.086062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6c5ab418cc406da83c6205382dd1ce'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.095263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6c5ab418cc406da83c6205382dd1ce'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.127074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 946/1889] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:17 2022] DELETE /identity/v3/users/8b6c5ab418cc406da83c6205382dd1ce => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:18.131533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:18.131986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:18.132093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:18.151482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:18.152677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:18.224750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 944/1890] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:18.234431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:18.236032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:18.299127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9598ede-9bfa-4bae-8ca7-8dbb5397053a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:18.305067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:18.306089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:18.306379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:18.306655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:18.325441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:18.328483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:18.346525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 947/1891] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:18.625440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:18.626909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:18.638901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d9511-7285-4f0d-9713-77edc03adf53 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:18.644376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:18.645160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:18.645371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:18.645644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] PATH_INFO: `/v3/projects/a863597cbf7c4fdabe033880aac0995b` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:18.652099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a863597cbf7c4fdabe033880aac0995b)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:18.654323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:18.705810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a863597cbf7c4fdabe033880aac0995b', 'request_id': 'req-a6775d37-f899-4de7-889c-098dde886fb9'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.712723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a863597cbf7c4fdabe033880aac0995b', 'request_id': 'req-a6775d37-f899-4de7-889c-098dde886fb9'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.719913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a863597cbf7c4fdabe033880aac0995b', 'request_id': 'req-a6775d37-f899-4de7-889c-098dde886fb9'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.731697 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] The token cache is being invalidate because project a863597cbf7c4fdabe033880aac0995b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:18.731989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.732633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.734159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 945/1892] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:18 2022] DELETE /identity/v3/projects/a863597cbf7c4fdabe033880aac0995b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:18.738125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:18.747586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91797e-a3fa-4dea-bc5d-d59474ac92e2 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:18.766364 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:18.806326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:18.806952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:18.807232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:18.807505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] PATH_INFO: `/v3/users/84066a219aec435580240e023e224dde` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:18.808632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84066a219aec435580240e023e224dde)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:18.810721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:18.859733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84066a219aec435580240e023e224dde'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.868646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84066a219aec435580240e023e224dde'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.869368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84066a219aec435580240e023e224dde'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.870017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84066a219aec435580240e023e224dde'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.877920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84066a219aec435580240e023e224dde'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:18.894881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 948/1893] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:18 2022] DELETE /identity/v3/users/84066a219aec435580240e023e224dde => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:19.168163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:19.169851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:19.199906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6775d37-f899-4de7-889c-098dde886fb9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:19.261236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:19.262017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:19.262308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:19.262583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] PATH_INFO: `/v3/projects/a056e7c38bea426bbbad7023d2ab9552` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:19.263751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a056e7c38bea426bbbad7023d2ab9552)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:19.266137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:19.298016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:19.317726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a056e7c38bea426bbbad7023d2ab9552', 'request_id': 'req-f52b1f87-d136-41fc-b92d-ff06162954d2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.326404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a056e7c38bea426bbbad7023d2ab9552', 'request_id': 'req-f52b1f87-d136-41fc-b92d-ff06162954d2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.333669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a056e7c38bea426bbbad7023d2ab9552', 'request_id': 'req-f52b1f87-d136-41fc-b92d-ff06162954d2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.359687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] The token cache is being invalidate because project a056e7c38bea426bbbad7023d2ab9552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:19.360157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.360913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.362718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 946/1894] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:19 2022] DELETE /identity/v3/projects/a056e7c38bea426bbbad7023d2ab9552 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:19.366532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:19.368105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:19.380989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758148d-b104-41ae-bc35-648bbb3228e8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:19.448583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:19.449238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:19.449516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:19.449803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] PATH_INFO: `/v3/users/303df38f2276453d87f9fe752a868c64` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:19.451016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=303df38f2276453d87f9fe752a868c64)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:19.453553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:19.498327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:19.501695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303df38f2276453d87f9fe752a868c64'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.511249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303df38f2276453d87f9fe752a868c64'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.512102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303df38f2276453d87f9fe752a868c64'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.512845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303df38f2276453d87f9fe752a868c64'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.522896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1fe31215-36e1-455e-9803-61bdb97430ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303df38f2276453d87f9fe752a868c64'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:19.561685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 949/1895] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:19 2022] DELETE /identity/v3/users/303df38f2276453d87f9fe752a868c64 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:19.855640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:19.857075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:19.877286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b1f87-d136-41fc-b92d-ff06162954d2 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:19.955225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:19.955869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:19.956145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:19.956416 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] PATH_INFO: `/v3/projects/4249abebce2d49cb85e01209c2001a3e` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:19.957529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4249abebce2d49cb85e01209c2001a3e)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:19.959510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.005807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4249abebce2d49cb85e01209c2001a3e', 'request_id': 'req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:20.015230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4249abebce2d49cb85e01209c2001a3e', 'request_id': 'req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:20.021679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4249abebce2d49cb85e01209c2001a3e', 'request_id': 'req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:20.034941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] The token cache is being invalidate because project 4249abebce2d49cb85e01209c2001a3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:20.035231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:20.035947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-632dca88-3ad1-481b-95fa-3a0c6ab43a2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:20.037584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 947/1896] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:19 2022] DELETE /identity/v3/projects/4249abebce2d49cb85e01209c2001a3e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:20.049538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.049538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.049538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.065064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:20.066378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:20.124829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 950/1897] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:02:20.133338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.133629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.133943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.154942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:20.156079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:20.164370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 948/1898] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:02:20.168542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.178305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675a47b-7e81-4e38-94bc-fd7a148931b9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.184260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.184884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.185165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.185438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.186127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:02:20.186923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.188634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.200090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 951/1899] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:02:20.205007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.214339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61439431-04a6-4b25-b2ea-c7e63448b4d3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.218918 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.219513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.219792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.220065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.220702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.222624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.250527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 949/1900] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:20 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:20.254788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.264666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af12b49-492d-4e16-9105-e7c9bb496f85 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.322287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.322894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.323175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.323443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.324074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.326030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.347333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 952/1901] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:20 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:02:20.351644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.363531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9bd70-1221-4db2-b470-69ce19b662ac admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.370638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.371334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.371529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.371802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.372443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.374095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.389563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 950/1902] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:02:20.392293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.408641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bce92-6e8d-4f3e-8141-dc20f3d82abc admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.415841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5304d-dac8-4c06-bcad-bf57b82891d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.416450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5304d-dac8-4c06-bcad-bf57b82891d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.416773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5304d-dac8-4c06-bcad-bf57b82891d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.417273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5304d-dac8-4c06-bcad-bf57b82891d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.417929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5304d-dac8-4c06-bcad-bf57b82891d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.419565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5304d-dac8-4c06-bcad-bf57b82891d7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.432070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 953/1903] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:20.436213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.442657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:20.446750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f309d5-89cc-4024-b013-0378ca72e04e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.453639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.454430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.454966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.455478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] PATH_INFO: `/v3/projects/345d1b9ec923491b8e7f4d77a20a576c/users/c3801be705a34f8aaafddcaa5c40b78c/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.471680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=345d1b9ec923491b8e7f4d77a20a576c, user_id=c3801be705a34f8aaafddcaa5c40b78c, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.474518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.489499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 951/1904] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:20 2022] PUT /identity/v3/projects/345d1b9ec923491b8e7f4d77a20a576c/users/c3801be705a34f8aaafddcaa5c40b78c/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:20.501736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.502046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.502327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.537179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] MFA Rules not processed for user `c3801be705a34f8aaafddcaa5c40b78c`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:20.538433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:20.552102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:20.612415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 954/1905] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:20.631557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:20.633359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:20.709091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75bb53a-c37d-4a97-9e65-222748f2d4d8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:20.712976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc104afe-a2c1-4d1c-b2d3-f1800df6d4ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:20.713618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104afe-a2c1-4d1c-b2d3-f1800df6d4ab service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:20.713922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104afe-a2c1-4d1c-b2d3-f1800df6d4ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:20.714229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc104afe-a2c1-4d1c-b2d3-f1800df6d4ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:20.723662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104afe-a2c1-4d1c-b2d3-f1800df6d4ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:20.725102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc104afe-a2c1-4d1c-b2d3-f1800df6d4ab service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:20.734265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 952/1906] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:30.606331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=20 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:30.729948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:49.008196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:49.009689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:49.079215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1fa84-4fe5-4ab4-b828-66aff4eb332c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:49.084006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:49.084689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:49.085211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:49.085399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] PATH_INFO: `/v3/users/8bd7eb1f2594444885b19052ae53ec0d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:49.086487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bd7eb1f2594444885b19052ae53ec0d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:49.088517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:49.174425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bd7eb1f2594444885b19052ae53ec0d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.187302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bd7eb1f2594444885b19052ae53ec0d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.187982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bd7eb1f2594444885b19052ae53ec0d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.188601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bd7eb1f2594444885b19052ae53ec0d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.199108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bd7eb1f2594444885b19052ae53ec0d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.226131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 955/1907] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:49 2022] DELETE /identity/v3/users/8bd7eb1f2594444885b19052ae53ec0d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:49.229530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:49.230107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:49.230553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:49.253893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:49.255372 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:49.356957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 953/1908] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:02:49.364489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:49.366259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:49.455063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035847a-43d1-4c86-b26c-f4f9f1dd6a49 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:49.460014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:49.460703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:49.461055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:49.461782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:49.472571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:49.474119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:49.490491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 956/1909] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:02:49.818490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:49.820666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:49.834751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ae506-4dd2-4a47-8315-cfb05170841f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:49.845175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:49.845830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:49.846202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:49.846475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] PATH_INFO: `/v3/projects/c98c34167630486fba99bbb0f10dfe0a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:49.847656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c98c34167630486fba99bbb0f10dfe0a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:49.849666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:49.901862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c98c34167630486fba99bbb0f10dfe0a', 'request_id': 'req-3253cf3f-54f1-4f74-9381-065fa374932b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.909345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c98c34167630486fba99bbb0f10dfe0a', 'request_id': 'req-3253cf3f-54f1-4f74-9381-065fa374932b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.914821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c98c34167630486fba99bbb0f10dfe0a', 'request_id': 'req-3253cf3f-54f1-4f74-9381-065fa374932b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.925642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] The token cache is being invalidate because project c98c34167630486fba99bbb0f10dfe0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:49.925922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.926580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:49.928073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 954/1910] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:49 2022] DELETE /identity/v3/projects/c98c34167630486fba99bbb0f10dfe0a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:49.932036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:49.942180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a7346-ac5a-45d9-bd25-6e7ac90a3885 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:49.961803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:50.001687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:50.002276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.002550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.002814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] PATH_INFO: `/v3/users/9d09e59f40cd4433aa0e9e70f178a50f` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.003908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d09e59f40cd4433aa0e9e70f178a50f)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:50.005797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:50.052852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d09e59f40cd4433aa0e9e70f178a50f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.059777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d09e59f40cd4433aa0e9e70f178a50f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.060390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d09e59f40cd4433aa0e9e70f178a50f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.060984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d09e59f40cd4433aa0e9e70f178a50f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.068501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e0aae40e-62b7-4b2f-94b9-9e4f38701913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d09e59f40cd4433aa0e9e70f178a50f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.085841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 957/1911] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:02:49 2022] DELETE /identity/v3/users/9d09e59f40cd4433aa0e9e70f178a50f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:50.356111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:50.358600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:50.377836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253cf3f-54f1-4f74-9381-065fa374932b admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:50.445180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:50.445843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.446160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.446439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] PATH_INFO: `/v3/projects/257dde259e7f4b179b53ce0dea2c87f0` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.447635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=257dde259e7f4b179b53ce0dea2c87f0)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:50.449615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:50.484342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:50.515337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257dde259e7f4b179b53ce0dea2c87f0', 'request_id': 'req-33ec0489-afd2-4bea-85eb-a2fa933151a7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.523700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257dde259e7f4b179b53ce0dea2c87f0', 'request_id': 'req-33ec0489-afd2-4bea-85eb-a2fa933151a7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.529873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257dde259e7f4b179b53ce0dea2c87f0', 'request_id': 'req-33ec0489-afd2-4bea-85eb-a2fa933151a7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.542264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] The token cache is being invalidate because project 257dde259e7f4b179b53ce0dea2c87f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:02:50.542555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.543200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-33ec0489-afd2-4bea-85eb-a2fa933151a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:02:50.544712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 955/1912] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:02:50 2022] DELETE /identity/v3/projects/257dde259e7f4b179b53ce0dea2c87f0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:50.550515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.550515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.550945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.574050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:50.575667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:50.659121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 958/1913] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:50.666800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174950b9-04c4-4314-b301-c3163da0053a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.667120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174950b9-04c4-4314-b301-c3163da0053a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.667350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174950b9-04c4-4314-b301-c3163da0053a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.701740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-174950b9-04c4-4314-b301-c3163da0053a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:50.703619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-174950b9-04c4-4314-b301-c3163da0053a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:50.715360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 956/1914] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:50.716290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:50.728879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f15c262-ddd9-4d7a-80db-8f19207242c8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:50.733940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:50.734584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.734858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.735138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.735776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:02:50.736070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:50.737837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:50.754590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 959/1915] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:02:50.758526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174950b9-04c4-4314-b301-c3163da0053a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:50.773045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174950b9-04c4-4314-b301-c3163da0053a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:50.787599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:50.788301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.788643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.788989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.789748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:50.792138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:50.839700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 957/1916] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:50 2022] POST /identity/v3/projects => generated 385 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:02:50.843497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:50.858414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ad8c9-c628-432b-b2b7-88e58d141ed6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:50.900017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:50.945983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:50.946655 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.946930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.947208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.947844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:50.949830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:50.974029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 960/1917] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:50 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:50.976061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:50.986890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95772d-715a-4199-88fe-c2a30347b803 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:50.991761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:50.992367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:50.992648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:50.992922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:50.993568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:50.995196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.006151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 958/1918] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:51.010757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.022128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975ae53-88bf-47a8-b4b3-cd23974c71cc admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.027254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf03940-c271-4c71-84e4-ae7be91363a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.027864 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf03940-c271-4c71-84e4-ae7be91363a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.028145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf03940-c271-4c71-84e4-ae7be91363a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.028420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf03940-c271-4c71-84e4-ae7be91363a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.029106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf03940-c271-4c71-84e4-ae7be91363a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.030969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf03940-c271-4c71-84e4-ae7be91363a9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.043138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 961/1919] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:51.048493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.058731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5dfe7-d9b4-407c-ba10-45877b9371b8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.063928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.064579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.064860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.065136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] PATH_INFO: `/v3/projects/1755912bd35d4041b9cbebba5baa14c5/users/f00aa99dcdbf47fd9d33611de7ec1ec7/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.079805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1755912bd35d4041b9cbebba5baa14c5, user_id=f00aa99dcdbf47fd9d33611de7ec1ec7, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.082513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.097186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 959/1920] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:51 2022] PUT /identity/v3/projects/1755912bd35d4041b9cbebba5baa14c5/users/f00aa99dcdbf47fd9d33611de7ec1ec7/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:02:51.108969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.109434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.109545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.151124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] MFA Rules not processed for user `f00aa99dcdbf47fd9d33611de7ec1ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:51.152404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:51.245285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 962/1921] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:51.250429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.324594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1792efe-d8d8-45bb-9fee-987a71457811 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.329336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.329953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.330273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.330545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.331183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.333095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.361650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 960/1922] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:51 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:51.365813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.386544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7054db4-7e26-4923-90e8-750d3e8957d1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.448283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.448799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.449072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.449336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.449955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.452529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.472987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 963/1923] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:51 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:51.476946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.487129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e53af5-fef1-4284-856b-d21350b28b45 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.491740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.492339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.492619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.492892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.493533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.495160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.506492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 961/1924] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:51.510171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.520920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20629d6c-8752-48ea-a27a-1e39dcd6ca09 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.525497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f1278-f279-4793-932f-367cb32409c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.526204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1278-f279-4793-932f-367cb32409c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.526417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1278-f279-4793-932f-367cb32409c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.526689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f1278-f279-4793-932f-367cb32409c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.527326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1278-f279-4793-932f-367cb32409c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.528907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f1278-f279-4793-932f-367cb32409c9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.538713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 964/1925] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:51.542966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.553185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0913f5-2395-457b-b720-06b09834400c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.557814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.558527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.558804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.559076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] PATH_INFO: `/v3/projects/241481c64cd144a89d4a6dc493f8babc/users/c36c83902c5c4c5a8c92a311edac4b2f/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.574743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241481c64cd144a89d4a6dc493f8babc, user_id=c36c83902c5c4c5a8c92a311edac4b2f, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.578136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.587615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:51.592525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 962/1926] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:51 2022] PUT /identity/v3/projects/241481c64cd144a89d4a6dc493f8babc/users/c36c83902c5c4c5a8c92a311edac4b2f/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:51.603977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.604478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.604867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.637096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] MFA Rules not processed for user `c36c83902c5c4c5a8c92a311edac4b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:51.638532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:51.643465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:51.714058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 965/1927] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:51.718473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.779581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2667989a-1fa8-43ce-942a-34d25040f3e8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.784202 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.784805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.785085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.785363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.785991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.787941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.815751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 963/1928] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:02:51 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:02:51.819876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.830592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6113b45a-a056-4adb-8288-e85dd4cf33c3 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.888617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.889220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.889523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.890114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.890459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.892628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.914017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 966/1929] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:02:51 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:02:51.917911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.939848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab61cc7-d3af-4c26-8520-ee55c99cb5a4 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.939848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.939848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.939848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.940141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.940141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.940141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:51.958237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 964/1930] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:02:51.962273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:51.972498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b88eb-f3ba-4222-a501-4c88c2b1d37a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:51.977194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:51.977842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:51.978152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:51.978422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] PATH_INFO: `/v3/projects/775d8d658a724356a8d5f06652b6e71a/users/29bb1c805b1a41d59b64b6a3af42fd78/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:51.994072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775d8d658a724356a8d5f06652b6e71a, user_id=29bb1c805b1a41d59b64b6a3af42fd78, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:51.996744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:52.011083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 967/1931] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:51 2022] PUT /identity/v3/projects/775d8d658a724356a8d5f06652b6e71a/users/29bb1c805b1a41d59b64b6a3af42fd78/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:02:52.015145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:52.025899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb29def-0c24-4af5-9f50-2d0bf46f9de8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:52.086620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25babfed-26c0-486e-86ce-7c483b7d725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:52.087205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25babfed-26c0-486e-86ce-7c483b7d725f admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:52.087510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25babfed-26c0-486e-86ce-7c483b7d725f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:52.087809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25babfed-26c0-486e-86ce-7c483b7d725f admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:52.088456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25babfed-26c0-486e-86ce-7c483b7d725f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:52.090073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25babfed-26c0-486e-86ce-7c483b7d725f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:52.103795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 965/1932] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:02:52.105323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:52.117659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3c09-5a26-487f-adb7-f77099d8e8a2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:52.122369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:52.123029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:52.123308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:52.125186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] PATH_INFO: `/v3/projects/775d8d658a724356a8d5f06652b6e71a/users/29bb1c805b1a41d59b64b6a3af42fd78/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:52.127262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=775d8d658a724356a8d5f06652b6e71a, user_id=29bb1c805b1a41d59b64b6a3af42fd78, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:52.129878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:52.151323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 968/1933] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:02:52 2022] PUT /identity/v3/projects/775d8d658a724356a8d5f06652b6e71a/users/29bb1c805b1a41d59b64b6a3af42fd78/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:02:52.153470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:52.153849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:52.154307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:52.185071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] MFA Rules not processed for user `29bb1c805b1a41d59b64b6a3af42fd78`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:02:52.186377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:52.256651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:52.268635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 966/1934] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:02:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:02:52.306160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:52.306160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:02:52.367208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bebb7-a8e2-4590-91f3-d8b1c00ad396 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:52.372382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:52.373462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:52.373804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:52.374116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:52.383522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:52.384953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:52.390217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:02:52.395971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 969/1935] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:02:52.908478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:02:52.917792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19543756-5217-4906-b22a-b8af803e69a2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:02:52.921451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:02:52.922177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:02:52.922524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:02:52.922868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:02:52.985428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:02:52.986933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:02:52.995904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 967/1936] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:02:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:02.991828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=30 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:06.609417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:06.611151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:06.622551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbd482-a2b5-4345-becd-6910f37d9e8a service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:06.626276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:06.626897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:06.627193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:06.627474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:06.705389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:06.706889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:06.720255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 970/1937] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:09.223875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.281907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.283544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.314619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380409c7-829e-49c9-aef0-c013ee3a3660 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.321213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.321869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.322196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.322471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] PATH_INFO: `/v3/users/f00aa99dcdbf47fd9d33611de7ec1ec7` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.323601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00aa99dcdbf47fd9d33611de7ec1ec7)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.325858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.359387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6f105-e1cd-425c-849d-059891cf7020 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.364440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.365130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.365477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.365813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] PATH_INFO: `/v3/users/27d1a24c9e15493195744bcbf598ac4a` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.367526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=27d1a24c9e15493195744bcbf598ac4a)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.369661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.386905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00aa99dcdbf47fd9d33611de7ec1ec7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.396587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00aa99dcdbf47fd9d33611de7ec1ec7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.397258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00aa99dcdbf47fd9d33611de7ec1ec7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.408496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00aa99dcdbf47fd9d33611de7ec1ec7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.409559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e5761fca-2018-4844-87be-0d1164b49af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00aa99dcdbf47fd9d33611de7ec1ec7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.421265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27d1a24c9e15493195744bcbf598ac4a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.429472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27d1a24c9e15493195744bcbf598ac4a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.430357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27d1a24c9e15493195744bcbf598ac4a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.431364 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27d1a24c9e15493195744bcbf598ac4a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.432334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 968/1938] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:09 2022] DELETE /identity/v3/users/f00aa99dcdbf47fd9d33611de7ec1ec7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:09.436730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.437105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.437583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.439161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27d1a24c9e15493195744bcbf598ac4a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.456347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 971/1939] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:09 2022] DELETE /identity/v3/users/27d1a24c9e15493195744bcbf598ac4a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:09.467060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:09.468906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.530489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 969/1940] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:09.540855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.542545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.556929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.559100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.565005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:09.575021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a8fac-33cd-4178-be33-b5ead8585e72 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.580520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.581165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.581564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.581851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] PATH_INFO: `/v3/users/cc952df3884241fbaca2223fdb2e8990` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.583060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc952df3884241fbaca2223fdb2e8990)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.585497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.601930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:09.616427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0c4c9-2c0c-49fe-849f-41f649a47fdc admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.621539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.622428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.622845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.623154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.633715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.635299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.646944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 972/1941] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:09.647118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc952df3884241fbaca2223fdb2e8990'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.647684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.649127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.657900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc952df3884241fbaca2223fdb2e8990'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.659318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc952df3884241fbaca2223fdb2e8990'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.668673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc952df3884241fbaca2223fdb2e8990'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.669543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6b1cc133-5071-4b6f-95c0-e2ef97c871fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc952df3884241fbaca2223fdb2e8990'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.673778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dab6b1-f197-4dd7-bebe-592882d443b7 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.680217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.680888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.681165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.681436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] PATH_INFO: `/v3/projects/aa84dfa1cc764dc99b6e804e778c3479` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.682839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa84dfa1cc764dc99b6e804e778c3479)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.684999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.693859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 970/1942] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:09 2022] DELETE /identity/v3/users/cc952df3884241fbaca2223fdb2e8990 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:09.698588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.698979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.699327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.720322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:09.721480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.744856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa84dfa1cc764dc99b6e804e778c3479', 'request_id': 'req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.751973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa84dfa1cc764dc99b6e804e778c3479', 'request_id': 'req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.761243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa84dfa1cc764dc99b6e804e778c3479', 'request_id': 'req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.778197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] The token cache is being invalidate because project aa84dfa1cc764dc99b6e804e778c3479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:09.778904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.779651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e509eb0b-9b07-4449-8f0f-31f6cd3ed021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:09.781943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 973/1943] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:09 2022] DELETE /identity/v3/projects/aa84dfa1cc764dc99b6e804e778c3479 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:09.790388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e8fa0-66ff-4b37-831c-b6c1aee7ed72 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.791061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e8fa0-66ff-4b37-831c-b6c1aee7ed72 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.791664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e8fa0-66ff-4b37-831c-b6c1aee7ed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.796966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 971/1944] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:09.807105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.811220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.819078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e52e8fa0-66ff-4b37-831c-b6c1aee7ed72 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:09.823000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e52e8fa0-66ff-4b37-831c-b6c1aee7ed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.875019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23742a-7cda-4206-a449-e571372d26ec None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.892609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 974/1945] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:09.900594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.901110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.901541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.925099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:09.926339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.927157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:09.934990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.935730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 975/1946] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:09.936297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.937061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.937395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.939917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.947885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.949318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.950391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4595b0-2e8a-440a-a141-64976cbcb0e8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.959682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.960040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 972/1947] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:09.960543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.960758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:09.960963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.961396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.962081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:09.962351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:09.962720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.964989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.975213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c689839-5dd8-4619-a270-2229c0781c9e service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:09.980074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:09.980711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:09.980988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:09.981317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] PATH_INFO: `/v3/projects/1755912bd35d4041b9cbebba5baa14c5` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:09.983451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1755912bd35d4041b9cbebba5baa14c5)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:09.985413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:09.986693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 976/1948] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:03:09.988129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.002881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfd143-c3d7-4d05-938c-f7f6c0c6e77d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.008538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.009268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.009423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.009699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.010620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.013549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.046304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1755912bd35d4041b9cbebba5baa14c5', 'request_id': 'req-36574e0a-0da9-4e02-a3ef-fab456136759'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.048618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 977/1949] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:09 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:10.053705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.055051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1755912bd35d4041b9cbebba5baa14c5', 'request_id': 'req-36574e0a-0da9-4e02-a3ef-fab456136759'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.063324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1755912bd35d4041b9cbebba5baa14c5', 'request_id': 'req-36574e0a-0da9-4e02-a3ef-fab456136759'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.075722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] The token cache is being invalidate because project 1755912bd35d4041b9cbebba5baa14c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:10.076078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:10.076193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.076952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.082322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 973/1950] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:09 2022] DELETE /identity/v3/projects/1755912bd35d4041b9cbebba5baa14c5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:10.087982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.089480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.165941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618fe45a-1032-4c38-ae9a-f14523ae2d8e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.180293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.180985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36574e0a-0da9-4e02-a3ef-fab456136759 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.181331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.182354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.182785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.183563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.187434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.188272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.188735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.188982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.189365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] PATH_INFO: `/v3/users/c36c83902c5c4c5a8c92a311edac4b2f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.191000 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36c83902c5c4c5a8c92a311edac4b2f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.195553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.217057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 978/1951] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:10 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:10.220946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.232033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.247411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f123336-aeb9-41ac-8885-f4e3a392de5e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.255848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.256568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.257382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.257382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.258252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.258367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36c83902c5c4c5a8c92a311edac4b2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.260191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.269798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36c83902c5c4c5a8c92a311edac4b2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.271108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36c83902c5c4c5a8c92a311edac4b2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.277733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 979/1952] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:10.282362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.287621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36c83902c5c4c5a8c92a311edac4b2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.288400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36c83902c5c4c5a8c92a311edac4b2f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.301619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bd6a9-95ae-40ea-a78d-7166be3f6bf5 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.307038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.307799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.308072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.308383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.309019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.310681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.311839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:10.326063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 980/1953] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:10.333408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 974/1954] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:10 2022] DELETE /identity/v3/users/c36c83902c5c4c5a8c92a311edac4b2f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:10.337743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.337893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.340284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.343895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.381094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a56a7b-9f64-4266-84db-d6758f0af602 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.439923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bc6dd-153a-4b6a-bbe8-6e169d88c519 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.456456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cf6da-5a3c-425a-b553-e274733375f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.456776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cf6da-5a3c-425a-b553-e274733375f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.457046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cf6da-5a3c-425a-b553-e274733375f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.457485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cf6da-5a3c-425a-b553-e274733375f5 admin admin] PATH_INFO: `/v3/projects/932c6d1e53f44d91b1a7da56d637b1de/users/41d85dde58db4a38aeccbf665340e126/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.481461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cf6da-5a3c-425a-b553-e274733375f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=932c6d1e53f44d91b1a7da56d637b1de, user_id=41d85dde58db4a38aeccbf665340e126, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.485167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cf6da-5a3c-425a-b553-e274733375f5 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.504958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 975/1955] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:10 2022] PUT /identity/v3/projects/932c6d1e53f44d91b1a7da56d637b1de/users/41d85dde58db4a38aeccbf665340e126/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:10.516693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.517854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.518242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.518520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] PATH_INFO: `/v3/projects/a6aebe98cbc14e818b977cb9e54e5b70` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.519854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.520350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.521130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.521900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6aebe98cbc14e818b977cb9e54e5b70)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.523959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.577882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] MFA Rules not processed for user `41d85dde58db4a38aeccbf665340e126`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:10.586540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.609219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6aebe98cbc14e818b977cb9e54e5b70', 'request_id': 'req-38c2259c-75ce-4d42-ad20-0955eb3a6305'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.620400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6aebe98cbc14e818b977cb9e54e5b70', 'request_id': 'req-38c2259c-75ce-4d42-ad20-0955eb3a6305'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.629760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6aebe98cbc14e818b977cb9e54e5b70', 'request_id': 'req-38c2259c-75ce-4d42-ad20-0955eb3a6305'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.657249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] The token cache is being invalidate because project a6aebe98cbc14e818b977cb9e54e5b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:10.657687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.658643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-38c2259c-75ce-4d42-ad20-0955eb3a6305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.659373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:10.663368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 981/1956] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:10 2022] DELETE /identity/v3/projects/a6aebe98cbc14e818b977cb9e54e5b70 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:10.669348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.670932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.671589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.725188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:10.728758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.780750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 976/1957] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:10.789927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.791489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.858345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 982/1958] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:10.859218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:10.861111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:10.909176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826b8a0-bf57-4e9a-b2bf-cdad7e12c57a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.915325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.916128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.916589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.916968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] PATH_INFO: `/v3/projects/241481c64cd144a89d4a6dc493f8babc` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.918160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241481c64cd144a89d4a6dc493f8babc)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.920217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.928905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:10.942723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4188e-3dea-49a0-b28d-313110d9f024 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:10.948296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473e33e-198c-4a69-aedd-ad7f8ac2cec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:10.948913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e33e-198c-4a69-aedd-ad7f8ac2cec7 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.949197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e33e-198c-4a69-aedd-ad7f8ac2cec7 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.949478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473e33e-198c-4a69-aedd-ad7f8ac2cec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.959217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473e33e-198c-4a69-aedd-ad7f8ac2cec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:10.960794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473e33e-198c-4a69-aedd-ad7f8ac2cec7 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:10.971559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 983/1959] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:10.971721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241481c64cd144a89d4a6dc493f8babc', 'request_id': 'req-1489da3a-2eb7-4eb3-b115-a74253618343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.973018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d637728-1672-40be-8421-a52c6861382f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:10.973313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d637728-1672-40be-8421-a52c6861382f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:10.973582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d637728-1672-40be-8421-a52c6861382f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:10.979747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241481c64cd144a89d4a6dc493f8babc', 'request_id': 'req-1489da3a-2eb7-4eb3-b115-a74253618343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.985984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241481c64cd144a89d4a6dc493f8babc', 'request_id': 'req-1489da3a-2eb7-4eb3-b115-a74253618343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:10.999600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3d637728-1672-40be-8421-a52c6861382f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:10.999600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3d637728-1672-40be-8421-a52c6861382f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.003873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] The token cache is being invalidate because project 241481c64cd144a89d4a6dc493f8babc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:11.004171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.004849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.006403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 977/1960] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:10 2022] DELETE /identity/v3/projects/241481c64cd144a89d4a6dc493f8babc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:11.010169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.011698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.102559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 984/1961] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:11.110575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d637728-1672-40be-8421-a52c6861382f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.111993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489da3a-2eb7-4eb3-b115-a74253618343 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.117722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.118421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.118701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.118981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] PATH_INFO: `/v3/users/29bb1c805b1a41d59b64b6a3af42fd78` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.120291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bb1c805b1a41d59b64b6a3af42fd78)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.122502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.125030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d637728-1672-40be-8421-a52c6861382f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.130267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.130880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.131161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.131434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.132147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:11.132425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.134065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.147122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 985/1962] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:03:11.151217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.162085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c58d26-5b74-46dc-8ff1-36ab3546f7c2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.173110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.174492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.174492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.174492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.174962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.176951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.191434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:11.192037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bb1c805b1a41d59b64b6a3af42fd78'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.203023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bb1c805b1a41d59b64b6a3af42fd78'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.204537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bb1c805b1a41d59b64b6a3af42fd78'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.217893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 986/1963] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:11 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:11.218786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.221431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bb1c805b1a41d59b64b6a3af42fd78'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.222401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bb1c805b1a41d59b64b6a3af42fd78'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.248560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 978/1964] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:11 2022] DELETE /identity/v3/users/29bb1c805b1a41d59b64b6a3af42fd78 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:11.249842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.255104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.284500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:11.335128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4336bf-e50e-4d79-89be-dcaab220c0f9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.351934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6496564-800b-4bb6-b1d2-708cadc80f9f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.357475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.358764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.359833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.359833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] PATH_INFO: `/v3/users/41d85dde58db4a38aeccbf665340e126` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.361197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d85dde58db4a38aeccbf665340e126)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.363297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.413218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.414164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.414448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.415103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.415529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.417922 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.424089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d85dde58db4a38aeccbf665340e126'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.435100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d85dde58db4a38aeccbf665340e126'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.436019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d85dde58db4a38aeccbf665340e126'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.445519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d85dde58db4a38aeccbf665340e126'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.446565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-851429e6-b8cf-4e3d-ba73-281aa4031ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d85dde58db4a38aeccbf665340e126'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.453233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 987/1965] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:11 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:11.457535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.459517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.465928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 979/1966] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:11 2022] DELETE /identity/v3/users/41d85dde58db4a38aeccbf665340e126 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:11.471608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.471976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.472328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.474140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f1151-ff27-4e5c-af50-3f5177893ea0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.497519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:11.499002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.540793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.541696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.542119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.542401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.543126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.544828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.557129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 988/1967] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:11.561018 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.572695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1168-8959-4e7a-a0d0-b339126438fa admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.576845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 980/1968] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:11.577729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.578449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.578724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.578991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.579691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.581425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.587210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.588636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.597963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 989/1969] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:11.599129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.614460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa14b3-d445-4a10-9a5b-6d279183cba5 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.619955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1636a-a3ba-4aec-888f-46867d134380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.620622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1636a-a3ba-4aec-888f-46867d134380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.621043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1636a-a3ba-4aec-888f-46867d134380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.621324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1636a-a3ba-4aec-888f-46867d134380 admin admin] PATH_INFO: `/v3/projects/4cda156116354d2fb92b8aade2c60c33/users/a010cca2d2b44c13bf61eedd7fec1282/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.632199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:11.636650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:11.646866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1636a-a3ba-4aec-888f-46867d134380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cda156116354d2fb92b8aade2c60c33, user_id=a010cca2d2b44c13bf61eedd7fec1282, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.649835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1636a-a3ba-4aec-888f-46867d134380 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.667007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 990/1970] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:11 2022] PUT /identity/v3/projects/4cda156116354d2fb92b8aade2c60c33/users/a010cca2d2b44c13bf61eedd7fec1282/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:11.670601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd68bb-ef4a-40c7-b9d2-6d4d246d93a6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.682722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.682965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.683159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.719497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] MFA Rules not processed for user `a010cca2d2b44c13bf61eedd7fec1282`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:11.720936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.738338 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.738993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.739277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.739655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.796276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 991/1971] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:11.807924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.813193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.828332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.835743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.851145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 981/1972] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:11.851991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:11.853580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:11.867174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c3f40-d4f9-4deb-9f3a-17697b9f27bc service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.875837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.877154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.877441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.877714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] PATH_INFO: `/v3/projects/775d8d658a724356a8d5f06652b6e71a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.879097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=775d8d658a724356a8d5f06652b6e71a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.881001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.911179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93607a-f008-4419-8da6-de0b59c04843 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:11.915021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b4d02-3ea8-43d3-9033-ad95118525dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:11.915666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b4d02-3ea8-43d3-9033-ad95118525dd service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.915926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b4d02-3ea8-43d3-9033-ad95118525dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.916196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b4d02-3ea8-43d3-9033-ad95118525dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:11.928324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b4d02-3ea8-43d3-9033-ad95118525dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:11.930459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b4d02-3ea8-43d3-9033-ad95118525dd service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:11.941214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 992/1973] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:11.945695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '775d8d658a724356a8d5f06652b6e71a', 'request_id': 'req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.953798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '775d8d658a724356a8d5f06652b6e71a', 'request_id': 'req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.968852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '775d8d658a724356a8d5f06652b6e71a', 'request_id': 'req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.981603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] The token cache is being invalidate because project 775d8d658a724356a8d5f06652b6e71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:11.981967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.982685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7679b9a-682a-41cb-b4ab-4e4376dfdfb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:11.985081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 982/1974] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:11 2022] DELETE /identity/v3/projects/775d8d658a724356a8d5f06652b6e71a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:11.991244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:11.991566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:11.991840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.023043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:12.024584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.127506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 993/1975] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:12.135594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.135892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.136139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.171826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:12.173387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.183491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 983/1976] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:12.189070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.200737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5554c92d-ccd6-4d93-8c47-305d89191235 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.212581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.218017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.218017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.218017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.218017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:12.218017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.218017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.235404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 994/1977] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:03:12.241558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.252875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cbc75-81e9-45a4-926b-95a77288fbbb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.260573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.261409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.261915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.262285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.263014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.266113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.296460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:12.318949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 984/1978] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/projects => generated 401 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:12.323206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.331165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.335035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.335847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab895-4a30-408a-b2d5-ca7161681cea admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.369140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:12.448017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31669482-57a8-4741-b189-8f0d5a34ce26 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.453229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.453779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.454094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.454375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] PATH_INFO: `/v3/projects/932c6d1e53f44d91b1a7da56d637b1de` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.455854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=932c6d1e53f44d91b1a7da56d637b1de)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.457462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.474438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.475332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.475627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.476109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.476755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.478775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.510045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 995/1979] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/users => generated 301 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:12.515131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.528303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfbdda-d9c3-40d9-b581-bd6e9cadeb96 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.544863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '932c6d1e53f44d91b1a7da56d637b1de', 'request_id': 'req-6624a070-0f87-4807-b6f4-6cc3e948e609'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:12.553963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '932c6d1e53f44d91b1a7da56d637b1de', 'request_id': 'req-6624a070-0f87-4807-b6f4-6cc3e948e609'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:12.560608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '932c6d1e53f44d91b1a7da56d637b1de', 'request_id': 'req-6624a070-0f87-4807-b6f4-6cc3e948e609'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:12.573190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] The token cache is being invalidate because project 932c6d1e53f44d91b1a7da56d637b1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:12.573886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:12.574612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6624a070-0f87-4807-b6f4-6cc3e948e609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:12.576143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 985/1980] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:12 2022] DELETE /identity/v3/projects/932c6d1e53f44d91b1a7da56d637b1de => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:12.581530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9531c9-0095-4261-b708-23f82462844a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.581858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9531c9-0095-4261-b708-23f82462844a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.582194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9531c9-0095-4261-b708-23f82462844a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.606755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-cd9531c9-0095-4261-b708-23f82462844a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:12.608819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cd9531c9-0095-4261-b708-23f82462844a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.610786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.611497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.611775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.612069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.612791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.614375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.620882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 986/1981] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:12.624742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 996/1982] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:12.628589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.628765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.628883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.629260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.630661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.643707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8f9bc-9266-4810-ba5b-99c562084ed7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.648808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.649508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.649873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.650238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.650947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.652825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.664502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:12.666074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 997/1983] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:12.666208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.671450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.675884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 987/1984] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:12.680792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.683726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af651c00-f19c-46f5-ba95-35d5f8fc39c2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.690544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26319527-1662-479b-aa1b-8fb5c3a74d90 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.692590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b78cc-c4ae-4b81-9c53-3cd13fa818d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.693528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b78cc-c4ae-4b81-9c53-3cd13fa818d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.693920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b78cc-c4ae-4b81-9c53-3cd13fa818d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.694299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b78cc-c4ae-4b81-9c53-3cd13fa818d7 admin admin] PATH_INFO: `/v3/projects/06b5b8c953474d87ac95063b975aa5d7/users/b284591b3821429a814c4850d4787c73/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.695779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.696446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.696719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.696991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.697828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:12.698981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.700593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.713038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b78cc-c4ae-4b81-9c53-3cd13fa818d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b5b8c953474d87ac95063b975aa5d7, user_id=b284591b3821429a814c4850d4787c73, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.719003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b78cc-c4ae-4b81-9c53-3cd13fa818d7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.721905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 988/1985] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:03:12.727053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.743340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eab933-4033-406f-97d1-7b184d99c01a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.757654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 998/1986] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:12 2022] PUT /identity/v3/projects/06b5b8c953474d87ac95063b975aa5d7/users/b284591b3821429a814c4850d4787c73/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:12.772676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.772966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.773245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.807374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] MFA Rules not processed for user `b284591b3821429a814c4850d4787c73`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:12.808699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:12.823938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.824487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.824869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.825015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.825645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.828115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.834660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:12.867024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 989/1987] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:12.871266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.873696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:12.904557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 999/1988] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:12.906273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.917618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9243b4-e9be-47e8-97a8-4b6c17bd6a37 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.943455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43c51e-6cbc-44a0-a179-3f3ca78c214d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.948211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.948890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.949169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.949435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.950164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.952985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.975527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 990/1989] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:12.979627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:12.980218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.980936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.981213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.981485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.982150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.984115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:12.989611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55162a-2866-41bd-8759-ad73062d0d8b admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:12.994251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:12.994870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:12.995152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:12.995423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:12.996060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:12.997633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.013523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 991/1990] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:13.015642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.016741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1000/1991] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:12 2022] POST /identity/v3/projects => generated 399 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:13.020662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.026157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cc41-a985-417c-84a3-d4792ace0e31 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.086885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315569b4-d702-4f02-8ef4-4a5da856f7d0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.089746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.090422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.090701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.090973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.091593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.091723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.092231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.092532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.092972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.093213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.093587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.095628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.106740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 992/1992] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:13.107656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.118949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e5be6-b0c9-42b9-b5a9-970ee553528d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.119518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1001/1993] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:13 2022] POST /identity/v3/users => generated 306 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:13.124359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.132801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388e900-053b-4679-860e-a19d73f1e8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.133738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388e900-053b-4679-860e-a19d73f1e8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.134066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388e900-053b-4679-860e-a19d73f1e8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.134343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388e900-053b-4679-860e-a19d73f1e8a3 admin admin] PATH_INFO: `/v3/projects/5fcbe6afec544c9f90f6a2ad10dbe9e7/users/a8cf4b8787014e42afde048308b3d708/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.147386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b0843-13b5-4189-a6e4-0ed6498a0431 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.151590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388e900-053b-4679-860e-a19d73f1e8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcbe6afec544c9f90f6a2ad10dbe9e7, user_id=a8cf4b8787014e42afde048308b3d708, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.153788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.154416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.154545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388e900-053b-4679-860e-a19d73f1e8a3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.154691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.154957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.155590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.157163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.168786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1002/1994] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:13.172053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 993/1995] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:13 2022] PUT /identity/v3/projects/5fcbe6afec544c9f90f6a2ad10dbe9e7/users/a8cf4b8787014e42afde048308b3d708/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:13.174028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.181514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.181800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.182108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.183631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a6a1c-f264-4665-9cee-0f9dd03f5cea admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.213964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] MFA Rules not processed for user `a8cf4b8787014e42afde048308b3d708`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:13.215224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:13.233008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:13.240098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:13.242775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.243435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.243712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.243982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] PATH_INFO: `/v3/projects/75092ed831234678bd4e737c0d13514b/users/de970fc688f44e709e41c36453b19885/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.259237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75092ed831234678bd4e737c0d13514b, user_id=de970fc688f44e709e41c36453b19885, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.262091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.276944 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1003/1996] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:13 2022] PUT /identity/v3/projects/75092ed831234678bd4e737c0d13514b/users/de970fc688f44e709e41c36453b19885/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:13.280887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.287214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 994/1997] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:13.291311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.301453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44f308-e342-4020-9fc9-1cba904d1ca2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.346013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f2ad1-e0ea-4a34-b32e-74e0581311f4 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.350710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.351335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.351616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.351888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.352530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.354145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.363622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.364026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1004/1998] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:13.364442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.364780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.365114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.365807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.367857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.368114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.378152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70574684-f0aa-48eb-9d90-586e84c408da admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.383494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfaf21-7954-4d15-bd3e-72cf790678b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.384216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfaf21-7954-4d15-bd3e-72cf790678b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.384503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfaf21-7954-4d15-bd3e-72cf790678b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.384961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfaf21-7954-4d15-bd3e-72cf790678b0 admin admin] PATH_INFO: `/v3/projects/75092ed831234678bd4e737c0d13514b/users/de970fc688f44e709e41c36453b19885/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.387415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfaf21-7954-4d15-bd3e-72cf790678b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75092ed831234678bd4e737c0d13514b, user_id=de970fc688f44e709e41c36453b19885, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.390917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dfaf21-7954-4d15-bd3e-72cf790678b0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.401482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 995/1999] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:13 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:13.403071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1005/2000] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:13 2022] PUT /identity/v3/projects/75092ed831234678bd4e737c0d13514b/users/de970fc688f44e709e41c36453b19885/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:13.407125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.413342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.413637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.413914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.443680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] MFA Rules not processed for user `de970fc688f44e709e41c36453b19885`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:13.444913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:13.474039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce616d8a-ef3c-41bd-8412-3d6c963d3799 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.482846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.484030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.484306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.484575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.485225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.487181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.512808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 996/2001] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:13 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:13.516805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.528954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072d470-8b85-4fec-bdf0-e92448d14dce admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.533805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1006/2002] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:13 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:13.534870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.535786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.536177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.536907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.537657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.539413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.550736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 997/2003] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:13.557139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.557139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:13.557446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.567885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb86e-902b-4538-9f8c-dc59e0e79352 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.574533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.575358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.575795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.576251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] PATH_INFO: `/v3/projects/9043421d934344c080b8b4ae3bb55905/users/09e8380d99724cd9a0b7403cb538e538/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.594719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9043421d934344c080b8b4ae3bb55905, user_id=09e8380d99724cd9a0b7403cb538e538, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.597720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.613795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 998/2004] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:13 2022] PUT /identity/v3/projects/9043421d934344c080b8b4ae3bb55905/users/09e8380d99724cd9a0b7403cb538e538/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:13.616214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:13.617731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.624423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:13.630240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017fbaa-2d67-4ded-af3e-6f835b1bc7e2 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.687990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af9e86-d0fb-4040-9a17-6e15a1525a0a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.693742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.694411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.694683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.694984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.695611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.697278 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.699095 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.700277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.700735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.701086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.708498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 999/2005] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:13.712846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.723456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981f6d9-d6a0-4274-901f-95d961474f15 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.731285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7d8ba-ee5f-4a0d-b642-73e51bf63fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.732022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d8ba-ee5f-4a0d-b642-73e51bf63fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.732299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d8ba-ee5f-4a0d-b642-73e51bf63fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.732570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7d8ba-ee5f-4a0d-b642-73e51bf63fe2 admin admin] PATH_INFO: `/v3/projects/9043421d934344c080b8b4ae3bb55905/users/09e8380d99724cd9a0b7403cb538e538/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.735303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7d8ba-ee5f-4a0d-b642-73e51bf63fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9043421d934344c080b8b4ae3bb55905, user_id=09e8380d99724cd9a0b7403cb538e538, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.737942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e7d8ba-ee5f-4a0d-b642-73e51bf63fe2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.750711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1000/2006] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:13 2022] PUT /identity/v3/projects/9043421d934344c080b8b4ae3bb55905/users/09e8380d99724cd9a0b7403cb538e538/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:13.766320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.766320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.766320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.790234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.792662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.798831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] MFA Rules not processed for user `09e8380d99724cd9a0b7403cb538e538`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:13.800051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:13.806780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1007/2007] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:13.892570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1001/2008] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:13.898916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.900388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:13.903683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:13.911924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e13d78-c45b-442d-ad77-247b12033b11 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.959369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff0a877-2578-424d-89e1-0ac42bc8c102 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:13.963050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.963663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.963946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.964225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.969108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:13.969736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:13.970039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:13.970341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:13.976027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:13.977444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:13.990805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1002/2009] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:14.039481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:14.041251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:14.052445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1008/2010] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:13 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:14.099718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:14.113872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6297a-4f32-44e0-925f-53eb7c454631 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:14.117129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc35a-5e4e-40fd-8bd8-a1365f686d38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:14.117756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc35a-5e4e-40fd-8bd8-a1365f686d38 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:14.118070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc35a-5e4e-40fd-8bd8-a1365f686d38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:14.118346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc35a-5e4e-40fd-8bd8-a1365f686d38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:14.132373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:14.182805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc35a-5e4e-40fd-8bd8-a1365f686d38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:14.184249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc35a-5e4e-40fd-8bd8-a1365f686d38 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:14.193232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1003/2011] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:14.800950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:14.802393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:14.809179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:14.867538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924d12c-5af8-4309-ba47-5067868fdd2b service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:14.872022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:14.872645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:14.872921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:14.873187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] PATH_INFO: `/v3/users/a010cca2d2b44c13bf61eedd7fec1282` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:14.874831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a010cca2d2b44c13bf61eedd7fec1282)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:14.876753 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:14.925071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a010cca2d2b44c13bf61eedd7fec1282'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:14.941784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a010cca2d2b44c13bf61eedd7fec1282'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:14.942434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a010cca2d2b44c13bf61eedd7fec1282'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:14.943041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a010cca2d2b44c13bf61eedd7fec1282'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:14.951043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a010cca2d2b44c13bf61eedd7fec1282'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:14.969390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1009/2012] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:14 2022] DELETE /identity/v3/users/a010cca2d2b44c13bf61eedd7fec1282 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:14.973785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:14.974298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:14.974392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:14.995142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:14.996341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:15.056144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1004/2013] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:15.066169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:15.067689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:15.151979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532b0b-fa6e-48a1-8665-ad3021c83cf0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:15.157957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a77b25-3dcf-4d9d-b00c-34afbd864483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:15.161161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a77b25-3dcf-4d9d-b00c-34afbd864483 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:15.161161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a77b25-3dcf-4d9d-b00c-34afbd864483 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:15.161161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a77b25-3dcf-4d9d-b00c-34afbd864483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:15.179833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a77b25-3dcf-4d9d-b00c-34afbd864483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:15.181522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a77b25-3dcf-4d9d-b00c-34afbd864483 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:15.202415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1010/2014] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:15.939344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:15.941054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:15.961692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68f811-3f16-488f-a7f7-8d96d3e00d5e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:15.968091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:15.969285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:15.969569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:15.969928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] PATH_INFO: `/v3/projects/4cda156116354d2fb92b8aade2c60c33` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:15.971369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cda156116354d2fb92b8aade2c60c33)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:15.973311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.025179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cda156116354d2fb92b8aade2c60c33', 'request_id': 'req-a0da1444-f17d-447e-9eed-ced52ae6c7a3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:16.032573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cda156116354d2fb92b8aade2c60c33', 'request_id': 'req-a0da1444-f17d-447e-9eed-ced52ae6c7a3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:16.038370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cda156116354d2fb92b8aade2c60c33', 'request_id': 'req-a0da1444-f17d-447e-9eed-ced52ae6c7a3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:16.049884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] The token cache is being invalidate because project 4cda156116354d2fb92b8aade2c60c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:16.050200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:16.050829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a0da1444-f17d-447e-9eed-ced52ae6c7a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:16.052297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1005/2015] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:15 2022] DELETE /identity/v3/projects/4cda156116354d2fb92b8aade2c60c33 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:16.057808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.058303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.058459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.078268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:16.079506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:16.100178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:16.144726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1011/2016] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:16.152174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.152460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.152759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.179512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:16.180818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:16.190355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1006/2017] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:16.200556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.205324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d50c21-c954-464f-9fe8-44513fff836a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.209964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.210622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.210903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.211173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.211811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:16.212110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.213690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.225179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1012/2018] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:03:16.229390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.239071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb29de-6441-4d1e-9f5d-e7f97cf8d537 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.243976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.244591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.244866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.245208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.245869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.247793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.276138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1007/2019] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:16 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:16.280300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.290266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3d5c-7097-412e-933e-6383f276a000 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.349270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.349884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.350194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.350466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.351104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.353026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.378437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1013/2020] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:16 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:16.382605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.394272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2c1d2-0488-4363-9177-4a2282dfe5ec admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.399154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.399765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.400042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.400308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.400989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.402631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.404985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:16.418492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1008/2021] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:16.422826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.431442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa57767-86ad-4da1-8d59-615255496958 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.436867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c3a4b-bbca-4c72-bb49-aac295b24fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.437487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3a4b-bbca-4c72-bb49-aac295b24fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.437790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3a4b-bbca-4c72-bb49-aac295b24fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.438182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c3a4b-bbca-4c72-bb49-aac295b24fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.438820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c3a4b-bbca-4c72-bb49-aac295b24fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.440437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64c3a4b-bbca-4c72-bb49-aac295b24fa5 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.451141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1014/2022] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:16.455385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.478183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c779d-0c6b-4b6a-bd20-2e738e62753c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.478183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.478183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.478183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.478582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] PATH_INFO: `/v3/projects/e2879f97af87438c8aacc1ffafba6a33/users/a19a9a380ca74cceb3284f1b005af0de/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.491456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2879f97af87438c8aacc1ffafba6a33, user_id=a19a9a380ca74cceb3284f1b005af0de, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.494797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.512532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1009/2023] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:16 2022] PUT /identity/v3/projects/e2879f97af87438c8aacc1ffafba6a33/users/a19a9a380ca74cceb3284f1b005af0de/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:16.526258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.526544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.526845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.556329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] MFA Rules not processed for user `a19a9a380ca74cceb3284f1b005af0de`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:16.557608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:16.621433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1015/2024] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:16.634017 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:16.635707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:16.697293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad58c37-4ea9-4370-8bd8-079d6644a9eb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:16.701062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf922a-f355-460e-ab99-328e3f53f87b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:16.701705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf922a-f355-460e-ab99-328e3f53f87b service glance] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:16.701983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf922a-f355-460e-ab99-328e3f53f87b service glance] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:16.702287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf922a-f355-460e-ab99-328e3f53f87b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:16.711095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf922a-f355-460e-ab99-328e3f53f87b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:16.712523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf922a-f355-460e-ab99-328e3f53f87b service glance] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:16.721265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1010/2025] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:26.618755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:26.717689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:31.249582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:31.251031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:31.319061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc37f6-5074-4daa-af7b-10fa0ddf5d73 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:31.323850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:31.324494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:31.324787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:31.325058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] PATH_INFO: `/v3/users/b284591b3821429a814c4850d4787c73` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:31.326387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b284591b3821429a814c4850d4787c73)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:31.328129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:31.389150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b284591b3821429a814c4850d4787c73'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:31.397625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b284591b3821429a814c4850d4787c73'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:31.398315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b284591b3821429a814c4850d4787c73'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:31.398976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b284591b3821429a814c4850d4787c73'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:31.406229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b284591b3821429a814c4850d4787c73'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:31.425502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1016/2026] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:31 2022] DELETE /identity/v3/users/b284591b3821429a814c4850d4787c73 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:31.430089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:31.430653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:31.430653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:31.450180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:31.451474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:31.511363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1011/2027] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:31.521835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:31.523278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:31.579654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafaa8e4-124c-46dd-8551-12f640f5f755 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:31.583401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:31.585067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:31.585067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:31.585067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:31.594824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:31.596271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:31.606429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1017/2028] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:31.957920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:31.959724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:31.976024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd1c5b-f5f2-4fbc-9668-ed1df753c2c9 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:31.981853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:31.982766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:31.983051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:31.983325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] PATH_INFO: `/v3/projects/06b5b8c953474d87ac95063b975aa5d7` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:31.984625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b5b8c953474d87ac95063b975aa5d7)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:31.987298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:32.041644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b5b8c953474d87ac95063b975aa5d7', 'request_id': 'req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.052331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b5b8c953474d87ac95063b975aa5d7', 'request_id': 'req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.058250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b5b8c953474d87ac95063b975aa5d7', 'request_id': 'req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.069599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] The token cache is being invalidate because project 06b5b8c953474d87ac95063b975aa5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:32.069898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.070616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.072112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1012/2029] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:31 2022] DELETE /identity/v3/projects/06b5b8c953474d87ac95063b975aa5d7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:32.076400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:32.086159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c0f9f1-d041-4864-8e84-5837306f8a17 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:32.104268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:32.146834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:32.147496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.147775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.148069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] PATH_INFO: `/v3/users/de970fc688f44e709e41c36453b19885` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.149193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=de970fc688f44e709e41c36453b19885)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:32.151145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:32.196855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de970fc688f44e709e41c36453b19885'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.204633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de970fc688f44e709e41c36453b19885'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.205242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de970fc688f44e709e41c36453b19885'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.205839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de970fc688f44e709e41c36453b19885'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.213014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de970fc688f44e709e41c36453b19885'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.230530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1018/2030] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:32 2022] DELETE /identity/v3/users/de970fc688f44e709e41c36453b19885 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:32.507846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:32.509488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:32.524409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc3ca5-5b8b-4eed-a7e9-6dd0c85e69d2 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:32.581745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:32.583266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:32.595637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:32.596274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.596552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.596827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] PATH_INFO: `/v3/projects/75092ed831234678bd4e737c0d13514b` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.597963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=75092ed831234678bd4e737c0d13514b)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:32.599896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:32.630916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:32.650945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295c480-1275-4e34-a37c-852c88adda0c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:32.655191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75092ed831234678bd4e737c0d13514b', 'request_id': 'req-66c920e6-4ef3-42c4-a008-56934f849a9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.664529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75092ed831234678bd4e737c0d13514b', 'request_id': 'req-66c920e6-4ef3-42c4-a008-56934f849a9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.672040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75092ed831234678bd4e737c0d13514b', 'request_id': 'req-66c920e6-4ef3-42c4-a008-56934f849a9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.685744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] The token cache is being invalidate because project 75092ed831234678bd4e737c0d13514b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:32.686064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.686743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66c920e6-4ef3-42c4-a008-56934f849a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.688262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1013/2031] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:32 2022] DELETE /identity/v3/projects/75092ed831234678bd4e737c0d13514b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:32.693146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab99be-104a-47f0-a175-19184316bcf9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.693438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab99be-104a-47f0-a175-19184316bcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.693704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab99be-104a-47f0-a175-19184316bcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.708945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:32.716070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-fbab99be-104a-47f0-a175-19184316bcf9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:32.717374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fbab99be-104a-47f0-a175-19184316bcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:32.726729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:32.727369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.727646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.727921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] PATH_INFO: `/v3/users/a8cf4b8787014e42afde048308b3d708` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.729046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8cf4b8787014e42afde048308b3d708)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:32.731006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:32.780678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8cf4b8787014e42afde048308b3d708'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.780888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1014/2032] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:32.789222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8cf4b8787014e42afde048308b3d708'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.789902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8cf4b8787014e42afde048308b3d708'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.790393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.790736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.790883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8cf4b8787014e42afde048308b3d708'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.791008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.798706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fce01d6f-fbbc-4834-9533-f3413ddc5fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8cf4b8787014e42afde048308b3d708'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:32.811028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:32.812325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:32.820639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1019/2033] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:32 2022] DELETE /identity/v3/users/a8cf4b8787014e42afde048308b3d708 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:32.822251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1015/2034] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:32.826577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:32.826751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.827222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.827408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.835639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bddd02-c830-4c2c-92e2-414866392ad6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:32.848502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:32.850025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:32.904528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:32.905405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.905710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.906030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.907682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:32.907682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:32.909352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:32.921703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1016/2035] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:03:32.925579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1020/2036] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:32.926159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:32.936157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:32.936650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09fa8b-698e-488a-b399-f625db41e051 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:32.937549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:32.941887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:32.942192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:32.942576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:32.943086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:32.944007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:32.945816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:32.976113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1017/2037] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:32 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:32.982862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.012907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e8ca5-cb37-4ab0-89d3-e238a0b988a0 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.020230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:33.040489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:33.071988 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa535ba-6b08-41ff-88f6-cbb1bc8bf667 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.073841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.074603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.074986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.075367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.077543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.079173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.079556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.079923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.080699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.082816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.086029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.087489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.102204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1021/2038] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:33.106745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1018/2039] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:32 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:33.110874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.125472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e7d18-7d96-4dd1-a007-8b8c524a8118 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.131829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.132712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.133455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.133843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.134765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.136507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.148265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1022/2040] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:33.152207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.167577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab1edf-64d2-4780-9ae6-7f3f925d3beb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.175177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7610c-4ccb-4d74-a34a-82648c097319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.175785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7610c-4ccb-4d74-a34a-82648c097319 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.176058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7610c-4ccb-4d74-a34a-82648c097319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.176358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7610c-4ccb-4d74-a34a-82648c097319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.177052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7610c-4ccb-4d74-a34a-82648c097319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.178913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7610c-4ccb-4d74-a34a-82648c097319 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.190633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1019/2041] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:33.194768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.204907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d640a-6eef-4c4a-bf73-03050db9e400 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.210285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.211142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.211427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.211722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] PATH_INFO: `/v3/projects/3e03a7811a764fada9e445e3041dfe89/users/e73029a4bc804be2a2d417da84507b66/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.227239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e03a7811a764fada9e445e3041dfe89, user_id=e73029a4bc804be2a2d417da84507b66, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.229792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.243145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1023/2042] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:33 2022] PUT /identity/v3/projects/3e03a7811a764fada9e445e3041dfe89/users/e73029a4bc804be2a2d417da84507b66/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:33.254740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.255026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.255406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.290315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] MFA Rules not processed for user `e73029a4bc804be2a2d417da84507b66`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:33.291583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:33.359022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1020/2043] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:33.372949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.374966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:33.395538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.397203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:33.462936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3eda1a-d816-4552-afe6-7dcfd49c84a6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.466841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.467496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.467790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.468062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.472217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668bd993-6d5a-4cac-85af-01571cfa4eb3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.476986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.477121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.477840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.478119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.478390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] PATH_INFO: `/v3/projects/5fcbe6afec544c9f90f6a2ad10dbe9e7` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.478501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.479504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fcbe6afec544c9f90f6a2ad10dbe9e7)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.481395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.488869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1024/2044] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:33.509239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.516377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:33.521430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7a84d-a52a-4866-bcd2-0045b07cd3ee service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.532912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fcbe6afec544c9f90f6a2ad10dbe9e7', 'request_id': 'req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.540356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fcbe6afec544c9f90f6a2ad10dbe9e7', 'request_id': 'req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.546280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fcbe6afec544c9f90f6a2ad10dbe9e7', 'request_id': 'req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.558104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] The token cache is being invalidate because project 5fcbe6afec544c9f90f6a2ad10dbe9e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:33.558395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.559130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.560813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1021/2045] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:33 2022] DELETE /identity/v3/projects/5fcbe6afec544c9f90f6a2ad10dbe9e7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:33.565339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.567153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:33.583614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.584249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.584556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.584799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] PATH_INFO: `/v3/users/e73029a4bc804be2a2d417da84507b66` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.586676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73029a4bc804be2a2d417da84507b66)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.589290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.632211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:33.635532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be6c7-f3e0-4522-8b7a-5e4d64388fb2 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.640575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.641750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.642087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.642350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73029a4bc804be2a2d417da84507b66'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.642489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] PATH_INFO: `/v3/users/09e8380d99724cd9a0b7403cb538e538` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.643848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e8380d99724cd9a0b7403cb538e538)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.645802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.656060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73029a4bc804be2a2d417da84507b66'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.657342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73029a4bc804be2a2d417da84507b66'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.657978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73029a4bc804be2a2d417da84507b66'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.665527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-164f081a-7b59-435d-8825-eab1393d559e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73029a4bc804be2a2d417da84507b66'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.687421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1025/2046] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:33 2022] DELETE /identity/v3/users/e73029a4bc804be2a2d417da84507b66 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:33.692818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.693101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.693375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.704190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e8380d99724cd9a0b7403cb538e538'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.714802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:33.715392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e8380d99724cd9a0b7403cb538e538'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.716064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:33.716240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e8380d99724cd9a0b7403cb538e538'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.724930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e8380d99724cd9a0b7403cb538e538'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.728127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e8380d99724cd9a0b7403cb538e538'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:33.753094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1022/2047] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:33 2022] DELETE /identity/v3/users/09e8380d99724cd9a0b7403cb538e538 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:33.787600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1026/2048] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:33.797885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:33.799492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:33.845710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:33.863858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4991b-7c15-4ef5-94a5-1b84cd755d24 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:33.867526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c7d36-4c1f-468b-aa20-ad53b196f3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:33.868144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c7d36-4c1f-468b-aa20-ad53b196f3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:33.868427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c7d36-4c1f-468b-aa20-ad53b196f3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:33.868703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c7d36-4c1f-468b-aa20-ad53b196f3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:33.934146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c7d36-4c1f-468b-aa20-ad53b196f3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:33.935581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c7d36-4c1f-468b-aa20-ad53b196f3b9 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:33.950964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1023/2049] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:34.108898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.111329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.139587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b417a-6f9b-4997-9af9-78e2d2b39576 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.158866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.159535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.159816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.160541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] PATH_INFO: `/v3/projects/9043421d934344c080b8b4ae3bb55905` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.161609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9043421d934344c080b8b4ae3bb55905)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.164114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.247890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9043421d934344c080b8b4ae3bb55905', 'request_id': 'req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.254392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9043421d934344c080b8b4ae3bb55905', 'request_id': 'req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.262820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9043421d934344c080b8b4ae3bb55905', 'request_id': 'req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.274764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] The token cache is being invalidate because project 9043421d934344c080b8b4ae3bb55905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:34.275097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.275686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.277249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1027/2050] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:34 2022] DELETE /identity/v3/projects/9043421d934344c080b8b4ae3bb55905 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:34.283879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7cd35-9ad2-440b-9d1e-52cb3f3e8183 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.284194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7cd35-9ad2-440b-9d1e-52cb3f3e8183 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.284474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7cd35-9ad2-440b-9d1e-52cb3f3e8183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.306891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-09c7cd35-9ad2-440b-9d1e-52cb3f3e8183 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:34.308115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-09c7cd35-9ad2-440b-9d1e-52cb3f3e8183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.324023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.325760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.375992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:34.382133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1024/2051] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:34.390370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.390664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.391474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.404511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d0e7f6-7243-41df-a0bc-efe4d1cfc0fa admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.410114 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.411374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.412090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.412399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] PATH_INFO: `/v3/projects/3e03a7811a764fada9e445e3041dfe89` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.413746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e03a7811a764fada9e445e3041dfe89)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.414710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:34.415785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.415946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.424651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1025/2052] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:34.428906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.438953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170378d-c845-4f55-b8fc-5605cfaf49d7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.444588 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.445325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.445649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.445942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.446962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:34.447313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.449166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.461968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1026/2053] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:03:34.464561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e03a7811a764fada9e445e3041dfe89', 'request_id': 'req-3452d823-45bc-4bd9-942c-30beda4d463e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.467103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.470955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e03a7811a764fada9e445e3041dfe89', 'request_id': 'req-3452d823-45bc-4bd9-942c-30beda4d463e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.477460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8c1b4-9975-4a30-8c37-2b6e4d74d1a9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.478932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e03a7811a764fada9e445e3041dfe89', 'request_id': 'req-3452d823-45bc-4bd9-942c-30beda4d463e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.490939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] The token cache is being invalidate because project 3e03a7811a764fada9e445e3041dfe89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:34.491225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.491896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3452d823-45bc-4bd9-942c-30beda4d463e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:34.493943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1028/2054] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:34 2022] DELETE /identity/v3/projects/3e03a7811a764fada9e445e3041dfe89 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:34.498597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26894713-9c34-423c-9bd7-f2a8a8690f52 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.498898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26894713-9c34-423c-9bd7-f2a8a8690f52 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.499166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26894713-9c34-423c-9bd7-f2a8a8690f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.519162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-26894713-9c34-423c-9bd7-f2a8a8690f52 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:34.520288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-26894713-9c34-423c-9bd7-f2a8a8690f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.536504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.537119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.537401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.537674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.538341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.540313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.554590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:34.571380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1027/2055] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:34.572236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.573816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.583037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1029/2056] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:34.590784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.591355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.591355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.612752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:34.613964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.649353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38bbaa-86aa-4a2f-9d14-b66372ce6ef9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.653935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.654663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.654948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.655216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.655847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.658433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.676956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1030/2057] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:34.681136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.684307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1028/2058] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:34.685425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.690139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec4609-3b09-4a60-bf2f-df8c5b87a178 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.694765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.695342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.695446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f70049-18a3-487d-8ebc-5fed2dda2bef admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.695617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.695882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.696514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:34.696808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.698413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.700155 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.700687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:34.701290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.701580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.701931 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.703165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.705282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.714349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1031/2059] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:03:34.716229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.718305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1029/2060] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:34.722382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.726869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77019bf7-c1c4-4477-8485-3fd33d118e8c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.731944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56e057-0b78-47dd-85ff-456b341fa4cc admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.732501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.733277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.733649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.734083 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.734801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.736668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.736797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.737383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.737668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.737943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.738619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.740355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.750702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1030/2061] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:34.754576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.764628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03157d-dca2-4a18-8e53-ae21571d45c5 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.770613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1032/2062] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:34.770736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c3205-a42e-47b8-a5b4-e1ded2580d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.770736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c3205-a42e-47b8-a5b4-e1ded2580d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.770932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c3205-a42e-47b8-a5b4-e1ded2580d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.771135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c3205-a42e-47b8-a5b4-e1ded2580d07 admin admin] PATH_INFO: `/v3/projects/de596ee46fbc4e8b861c61daa9cbdbba/users/48618cc0f4ad4252aac783df2a87b2c5/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.771567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.787165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c3205-a42e-47b8-a5b4-e1ded2580d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de596ee46fbc4e8b861c61daa9cbdbba, user_id=48618cc0f4ad4252aac783df2a87b2c5, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.789909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c3205-a42e-47b8-a5b4-e1ded2580d07 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.802858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1031/2063] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:34 2022] PUT /identity/v3/projects/de596ee46fbc4e8b861c61daa9cbdbba/users/48618cc0f4ad4252aac783df2a87b2c5/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:34.814825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745228-2099-4e40-83be-168844ba4aec None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.815112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745228-2099-4e40-83be-168844ba4aec None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.815385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6745228-2099-4e40-83be-168844ba4aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.835124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ef3ca-917f-46e7-b2f1-d65505d6d269 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.842572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f6745228-2099-4e40-83be-168844ba4aec None None] MFA Rules not processed for user `48618cc0f4ad4252aac783df2a87b2c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:34.843789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f6745228-2099-4e40-83be-168844ba4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.892074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.892712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.893243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.893516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.894204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.896157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.908800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1032/2064] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:34.918593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1033/2065] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:34 2022] POST /identity/v3/users => generated 287 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:34.920948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745228-2099-4e40-83be-168844ba4aec None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.922374 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f6745228-2099-4e40-83be-168844ba4aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:34.922507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.932105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2cfb3-438c-48d0-a2ae-b877b40bf433 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.936738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.937317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.937615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.938672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.939587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.940690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.950629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1034/2066] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:34.955116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.956316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:34.964859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a3fd4-aa14-45de-9173-c53179e35dbe admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.969566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.970190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.970471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.970746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.971375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:34.972966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:34.983466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1035/2067] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:34.987636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:34.988424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6745228-2099-4e40-83be-168844ba4aec None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:34.992400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055c7d6-657c-41aa-a509-6e3e1c904818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:34.993007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055c7d6-657c-41aa-a509-6e3e1c904818 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:34.993282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055c7d6-657c-41aa-a509-6e3e1c904818 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:34.993549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055c7d6-657c-41aa-a509-6e3e1c904818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:34.997801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919e9f7-8f49-41b3-9c6e-060d3baf4d83 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:35.002915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055c7d6-657c-41aa-a509-6e3e1c904818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:35.003039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:35.003574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.003842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.004126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] PATH_INFO: `/v3/projects/041be21194ef4e34bd6aed80d16a2889/users/1982127ad7904b568da4ebe4af6a5a38/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.004343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055c7d6-657c-41aa-a509-6e3e1c904818 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:35.013110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1033/2068] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:35.019587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=041be21194ef4e34bd6aed80d16a2889, user_id=1982127ad7904b568da4ebe4af6a5a38, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:35.022306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:35.035744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1036/2069] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:34 2022] PUT /identity/v3/projects/041be21194ef4e34bd6aed80d16a2889/users/1982127ad7904b568da4ebe4af6a5a38/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:35.046876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.047232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.047509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.075745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] MFA Rules not processed for user `1982127ad7904b568da4ebe4af6a5a38`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:35.076960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:35.151364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1034/2070] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:35.199639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:35.203786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:35.233047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:35.284534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb13a7-4aab-4447-9121-50c9c063f35c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:35.284534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de2a5-34e6-4861-8883-6d2ab2c0dd09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:35.284534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de2a5-34e6-4861-8883-6d2ab2c0dd09 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.284534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de2a5-34e6-4861-8883-6d2ab2c0dd09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.285032 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de2a5-34e6-4861-8883-6d2ab2c0dd09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.298891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de2a5-34e6-4861-8883-6d2ab2c0dd09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:35.300700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de2a5-34e6-4861-8883-6d2ab2c0dd09 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:35.318649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1037/2071] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:35.350035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:35.426877 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37053e35-bd71-4d54-9eef-b890808fc45c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:35.431483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:35.432122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.432402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.432679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] PATH_INFO: `/v3/users/1982127ad7904b568da4ebe4af6a5a38` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.433822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1982127ad7904b568da4ebe4af6a5a38)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:35.435779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:35.482045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1982127ad7904b568da4ebe4af6a5a38'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:35.489388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1982127ad7904b568da4ebe4af6a5a38'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:35.490025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1982127ad7904b568da4ebe4af6a5a38'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:35.495194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:35.497330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1982127ad7904b568da4ebe4af6a5a38'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:35.498318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1982127ad7904b568da4ebe4af6a5a38'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:35.517546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1035/2072] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:35 2022] DELETE /identity/v3/users/1982127ad7904b568da4ebe4af6a5a38 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:35.522256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.522539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.522885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.541919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:35.543143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:35.600481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1038/2073] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:35.610635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:35.612070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:35.685871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b1fc1-f794-4d8f-83e6-73ff3b1ea8e0 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:35.689536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2077077-e655-4b38-8a47-58f9825f2b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:35.690181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2077077-e655-4b38-8a47-58f9825f2b4c service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.690457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2077077-e655-4b38-8a47-58f9825f2b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.690728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2077077-e655-4b38-8a47-58f9825f2b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.699505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2077077-e655-4b38-8a47-58f9825f2b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:35.700933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2077077-e655-4b38-8a47-58f9825f2b4c service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:35.710199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1036/2074] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:35.967381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:35.968883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:35.984878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec3ca0-2afb-4e94-97c1-43d06a082d6f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:35.990293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:35.990979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:35.991260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:35.991575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] PATH_INFO: `/v3/projects/041be21194ef4e34bd6aed80d16a2889` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:35.992681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=041be21194ef4e34bd6aed80d16a2889)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:35.994684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.042573 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '041be21194ef4e34bd6aed80d16a2889', 'request_id': 'req-15a65020-9395-441f-b919-d499749d9c9b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:36.048700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '041be21194ef4e34bd6aed80d16a2889', 'request_id': 'req-15a65020-9395-441f-b919-d499749d9c9b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:36.057248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '041be21194ef4e34bd6aed80d16a2889', 'request_id': 'req-15a65020-9395-441f-b919-d499749d9c9b'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:36.068428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] The token cache is being invalidate because project 041be21194ef4e34bd6aed80d16a2889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:36.068738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:36.069370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-15a65020-9395-441f-b919-d499749d9c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:36.070926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1039/2075] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:35 2022] DELETE /identity/v3/projects/041be21194ef4e34bd6aed80d16a2889 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:03:36.075664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.075955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.076235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.097587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:36.098834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:36.157176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1037/2076] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:36.164962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.165251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.165556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.186129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:36.187323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:36.195300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1040/2077] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:36.199420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.209579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346deb-898b-4991-be62-66db4b406ff3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.214543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.215145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.215424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.215834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.216337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:36.216635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.218253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.230582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1038/2078] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:03:36.234807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.243917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768658f-841d-4df5-85a4-b4dc8d735ebc None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.248579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.249165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.249436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.249701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.250391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.252262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.282150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1041/2079] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:36.283504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.294012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b56a39-2680-4a1c-8e21-94b65344e62c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.352129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.353248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.353542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.353895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.354571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.356669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.377187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:36.387737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.388616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1039/2080] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:36.407805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0ba56-b206-42c0-b33d-a34ab5007de8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.412443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.413050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.413327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.413594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.414256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.415899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.426673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1042/2081] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:36.430763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.441325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1355a-0861-4d75-b24e-3ad7d11d1d28 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.445962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f15403-01ba-458a-8ccb-3984057f7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.446592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15403-01ba-458a-8ccb-3984057f7055 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.446868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15403-01ba-458a-8ccb-3984057f7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.447137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15403-01ba-458a-8ccb-3984057f7055 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.447774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f15403-01ba-458a-8ccb-3984057f7055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.449348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f15403-01ba-458a-8ccb-3984057f7055 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.460298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1040/2082] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:36.464313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.474325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8e285-6eb6-4d99-ac70-a93a15c61a85 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.479286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.479930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.480205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.480471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] PATH_INFO: `/v3/projects/22c38ccf2f8e4ff1b45e9f8dcc1a7cbc/users/d82d3660a1cd4c25a906bba54b869bb0/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.489039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:36.498322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c38ccf2f8e4ff1b45e9f8dcc1a7cbc, user_id=d82d3660a1cd4c25a906bba54b869bb0, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.502043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.516826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1043/2083] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:36 2022] PUT /identity/v3/projects/22c38ccf2f8e4ff1b45e9f8dcc1a7cbc/users/d82d3660a1cd4c25a906bba54b869bb0/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:36.527355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.527713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.528056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.555261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] MFA Rules not processed for user `d82d3660a1cd4c25a906bba54b869bb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:36.556470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:36.620271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1041/2084] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:36.624804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.687898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc1169-b4b1-4211-aca2-4c6ef1568d21 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.693048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.693651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.693928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.694230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.694871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.696776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.723697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1044/2085] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:36.727846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.738867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318f6a9-c7fe-44cb-baa1-7ca217ac64b7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.794263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.794862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.795139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.795410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.796026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.798435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.819259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1042/2086] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:36.823272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.833419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22647e0-4727-4d2b-ae4d-79ee64f5a39d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.837973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.838597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.838875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.839145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.839780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.841356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.851594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1045/2087] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:36.855318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.865604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4539f-2526-45be-bba5-e92b9269a046 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.870566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368eeacd-2eda-45e3-ab32-7ff3c2e3d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.871099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368eeacd-2eda-45e3-ab32-7ff3c2e3d90c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.871378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368eeacd-2eda-45e3-ab32-7ff3c2e3d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.871668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368eeacd-2eda-45e3-ab32-7ff3c2e3d90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.872282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368eeacd-2eda-45e3-ab32-7ff3c2e3d90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.873855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368eeacd-2eda-45e3-ab32-7ff3c2e3d90c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.884242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1043/2088] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:36.888267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:36.898354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6232f4-61a5-48a1-a15e-5397d0f66687 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:36.903103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:36.903776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:36.904050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:36.904320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] PATH_INFO: `/v3/projects/0a8e13da74b94aff8f92cb462613a7bb/users/c1e598e165f64a6da5d538b0c0b2f334/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:36.918519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a8e13da74b94aff8f92cb462613a7bb, user_id=c1e598e165f64a6da5d538b0c0b2f334, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:36.921172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:36.933841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1046/2089] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:36 2022] PUT /identity/v3/projects/0a8e13da74b94aff8f92cb462613a7bb/users/c1e598e165f64a6da5d538b0c0b2f334/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:37.000543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.000831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.001111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.029393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] MFA Rules not processed for user `c1e598e165f64a6da5d538b0c0b2f334`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:37.030633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:37.073598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:37.094628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1044/2090] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:37.099060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.167293 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47ebcc-9ba5-45da-8a23-a16057dc0373 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.173391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.174113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.174465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.175329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.175712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.177671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.207423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1047/2091] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:37 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:03:37.211469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.222977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7cf32-8a0b-4a41-a4ce-135ade78186d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.285238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.285941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.286306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.286674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.287378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.289426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.314074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1045/2092] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:37 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:37.316367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.326610 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb84221-a81b-4892-bccc-ebe2738283d6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.331294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.331890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.332161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.332427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.333052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.334732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.345793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1048/2093] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:03:37.349904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.360341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6236672d-78bf-4bb9-9200-642aa2595c40 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.364890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.365527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.365802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.366308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] PATH_INFO: `/v3/projects/f32d20be52124aa9beb62bbcf5e17f43/users/09b7c920e6db4b0e96d132cc541b33b6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.380390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32d20be52124aa9beb62bbcf5e17f43, user_id=09b7c920e6db4b0e96d132cc541b33b6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.383075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.396533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1046/2094] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:37 2022] PUT /identity/v3/projects/f32d20be52124aa9beb62bbcf5e17f43/users/09b7c920e6db4b0e96d132cc541b33b6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:37.400499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.411417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23623a1-f2ad-4363-bed9-98ff1c929a67 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.420157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:37.482367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b54b9-cbb6-4715-8160-03f93b95ade2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.482980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b54b9-cbb6-4715-8160-03f93b95ade2 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.483256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b54b9-cbb6-4715-8160-03f93b95ade2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.483526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b54b9-cbb6-4715-8160-03f93b95ade2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.484163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b54b9-cbb6-4715-8160-03f93b95ade2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.485792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b54b9-cbb6-4715-8160-03f93b95ade2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.498508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1049/2095] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:37.499628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.509721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2b8af-59d9-4490-9dc6-2d2f029c9382 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.515745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.516898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.517179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.517452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] PATH_INFO: `/v3/projects/f32d20be52124aa9beb62bbcf5e17f43/users/09b7c920e6db4b0e96d132cc541b33b6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.519641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f32d20be52124aa9beb62bbcf5e17f43, user_id=09b7c920e6db4b0e96d132cc541b33b6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.522306 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.533365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1047/2096] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:37 2022] PUT /identity/v3/projects/f32d20be52124aa9beb62bbcf5e17f43/users/09b7c920e6db4b0e96d132cc541b33b6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:37.543794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.544079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.548698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.575499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] MFA Rules not processed for user `09b7c920e6db4b0e96d132cc541b33b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:37.576742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:37.645604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1050/2097] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:37.657960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:37.660240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:37.729142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd6b65-267f-471f-a04d-5e3f5632d437 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:37.732835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:37.733448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:37.733738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:37.733984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:37.796103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:37.807770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:37.809078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:37.821061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1048/2098] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:40.283458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:40.284378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:40.382513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5294b-a6a0-4d54-b8f4-5afd89b2a74a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:40.390339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:40.391550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:40.393511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:40.393880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] PATH_INFO: `/v3/users/48618cc0f4ad4252aac783df2a87b2c5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:40.398735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=48618cc0f4ad4252aac783df2a87b2c5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:40.400983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:40.441294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:40.442842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:40.458020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea5d65-b1a4-4eae-8861-537098a9c873 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:40.463348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:40.468380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:40.468673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:40.468919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:40.472270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:40.496001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48618cc0f4ad4252aac783df2a87b2c5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:40.505487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48618cc0f4ad4252aac783df2a87b2c5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:40.507028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48618cc0f4ad4252aac783df2a87b2c5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:40.508295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48618cc0f4ad4252aac783df2a87b2c5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:40.520942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2001e125-2e5c-4c65-a363-82ab922df9fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48618cc0f4ad4252aac783df2a87b2c5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:40.544310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1051/2099] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:40 2022] DELETE /identity/v3/users/48618cc0f4ad4252aac783df2a87b2c5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:40.549507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:40.549977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:40.550430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:40.578640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:40.580042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:40.580413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:40.581486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:40.591489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1049/2100] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:40.658359 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1052/2101] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:40.667832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:40.669359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:40.739281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cabab-87f9-4604-a7cb-e89a7c6115c1 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:40.796367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:40.796889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:40.797830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:40.798200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:40.807908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:40.809604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:40.820529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1050/2102] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:03:41.119342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.122276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.146053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bb937-fcbb-40c0-8a0e-bea6ee1db2a9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.151783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.152503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.152842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.153176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] PATH_INFO: `/v3/projects/de596ee46fbc4e8b861c61daa9cbdbba` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.154398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] RBAC: Authorizing `identity:delete_project(project_id=de596ee46fbc4e8b861c61daa9cbdbba)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.156416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.206351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.206505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de596ee46fbc4e8b861c61daa9cbdbba', 'request_id': 'req-2dbcce9e-1d6f-44c6-8198-b8c60741800e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:41.207899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.212090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de596ee46fbc4e8b861c61daa9cbdbba', 'request_id': 'req-2dbcce9e-1d6f-44c6-8198-b8c60741800e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:41.219411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de596ee46fbc4e8b861c61daa9cbdbba', 'request_id': 'req-2dbcce9e-1d6f-44c6-8198-b8c60741800e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:41.231949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] The token cache is being invalidate because project de596ee46fbc4e8b861c61daa9cbdbba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:41.232306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:41.232941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2dbcce9e-1d6f-44c6-8198-b8c60741800e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:41.234675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1053/2103] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:41 2022] DELETE /identity/v3/projects/de596ee46fbc4e8b861c61daa9cbdbba => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:41.239182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c167f26-db08-4b6d-9f4f-989da43dd523 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.239465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c167f26-db08-4b6d-9f4f-989da43dd523 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.239743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c167f26-db08-4b6d-9f4f-989da43dd523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.258557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2c167f26-db08-4b6d-9f4f-989da43dd523 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:41.259763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2c167f26-db08-4b6d-9f4f-989da43dd523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.264187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:41.269492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa96a1-65c6-4ce3-adb5-61ff042f6202 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.292366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:41.317765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.318331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.318607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.318898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.320609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.321566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1054/2104] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:41.330122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.330403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.330839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.351943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:41.353138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.361112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1055/2105] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:41.364896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.373706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276f1132-d23f-4cff-8a08-993caef4a571 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.378358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.378933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.379212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.379484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.380119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:03:41.380437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.382028 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.383382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.384707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.394518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1056/2106] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:03:41.395435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1051/2107] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:41.398859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.409528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00754e3b-eb7c-4a5a-9fb4-5bc91bf77a8a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.415423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.416024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.416307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.416582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.417225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.419199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.451650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1057/2108] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:03:41 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:41.456197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.467116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e7117e-e491-4e8f-bb13-d7e9be5906da service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.526536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.527272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.527550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.527827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.528484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.530436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.553468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1052/2109] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:03:41 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:03:41.557267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.568958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7978c2-111c-4e94-aac5-8c2ecbeae563 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.574287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.574771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.575047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.575389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.575943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.577517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.590031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1058/2110] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:41.594525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.605580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f3c03-a409-47a0-9f75-1bb621f7a0a5 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.610538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84f911-7353-4a6f-9b99-2ee7c7917634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.611241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84f911-7353-4a6f-9b99-2ee7c7917634 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.611583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84f911-7353-4a6f-9b99-2ee7c7917634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.611923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84f911-7353-4a6f-9b99-2ee7c7917634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.612632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84f911-7353-4a6f-9b99-2ee7c7917634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.614457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84f911-7353-4a6f-9b99-2ee7c7917634 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.630065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1053/2111] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:03:41.631903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.648238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca39726-bcc8-4c24-9897-9d1cf38ebb61 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.653860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.655209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.655489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.655768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] PATH_INFO: `/v3/projects/cb7a4ece4b1c447dab3ba43e93dc7903/users/f312fd0cb75541099871236e6dd6ffac/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.684887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7a4ece4b1c447dab3ba43e93dc7903, user_id=f312fd0cb75541099871236e6dd6ffac, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.688890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.703393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1059/2112] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:03:41 2022] PUT /identity/v3/projects/cb7a4ece4b1c447dab3ba43e93dc7903/users/f312fd0cb75541099871236e6dd6ffac/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:41.714355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.714645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.714921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.747066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] MFA Rules not processed for user `f312fd0cb75541099871236e6dd6ffac`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:41.748302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.769467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:41.814060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1054/2113] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:03:41.824842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:41.826562 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:41.881334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1643fa-f50e-4a3a-aa77-6c1e7a19a6ec admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:41.884940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:41.885527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:41.885803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:41.886100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:41.895136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:41.896544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:41.905245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1060/2114] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:50.184740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:50.186145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:50.254460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453b32f-7806-4a1b-8ec5-28be8abe6872 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:50.259669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:50.260280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:50.260579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:50.260853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:50.262622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:50.274051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:50.275644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891280e-ceb6-476f-bfdd-715b53f0cf7a service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:50.285184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1055/2115] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:51.901214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:52.711972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:52.713451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:52.778383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2be9c-3202-4d0c-b53b-f5f9a9426c07 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:52.783540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:52.784158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:52.784439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:52.784712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] PATH_INFO: `/v3/users/d82d3660a1cd4c25a906bba54b869bb0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:52.785831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d82d3660a1cd4c25a906bba54b869bb0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:52.788176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:52.836945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd82d3660a1cd4c25a906bba54b869bb0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:52.844252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd82d3660a1cd4c25a906bba54b869bb0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:52.844873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd82d3660a1cd4c25a906bba54b869bb0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:52.845516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd82d3660a1cd4c25a906bba54b869bb0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:52.852528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd82d3660a1cd4c25a906bba54b869bb0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:52.870437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1061/2116] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:52 2022] DELETE /identity/v3/users/d82d3660a1cd4c25a906bba54b869bb0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:52.874625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:52.875094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:52.875964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:52.896613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:52.897733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:52.963028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1056/2117] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:52.972657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:52.974085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:53.036591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26378ef-0a08-4e3f-adb6-e31652887a6c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:53.040534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:53.041252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:53.041533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:53.041802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:53.052508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:53.053980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:53.065124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1062/2118] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:03:53.446599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:53.448450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:53.472524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa616-4e77-4d0a-9864-b4034d60b6f3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:53.477497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:53.478135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:53.478419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:53.478691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] PATH_INFO: `/v3/projects/22c38ccf2f8e4ff1b45e9f8dcc1a7cbc` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:53.480433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22c38ccf2f8e4ff1b45e9f8dcc1a7cbc)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:53.482431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:53.528965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:53.530824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22c38ccf2f8e4ff1b45e9f8dcc1a7cbc', 'request_id': 'req-18d04627-246b-41bc-9840-cc5f0d6bdf3b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.540055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22c38ccf2f8e4ff1b45e9f8dcc1a7cbc', 'request_id': 'req-18d04627-246b-41bc-9840-cc5f0d6bdf3b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.547128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22c38ccf2f8e4ff1b45e9f8dcc1a7cbc', 'request_id': 'req-18d04627-246b-41bc-9840-cc5f0d6bdf3b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.558788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] The token cache is being invalidate because project 22c38ccf2f8e4ff1b45e9f8dcc1a7cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:53.559192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.559778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.561484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1057/2119] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:53 2022] DELETE /identity/v3/projects/22c38ccf2f8e4ff1b45e9f8dcc1a7cbc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:53.567487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:53.577119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8f760-2fa0-40d0-9efa-7773d516713b service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:53.603738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:53.646630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:53.647270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:53.647553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:53.647826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] PATH_INFO: `/v3/users/c1e598e165f64a6da5d538b0c0b2f334` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:53.648974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1e598e165f64a6da5d538b0c0b2f334)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:53.650949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:53.706877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1e598e165f64a6da5d538b0c0b2f334'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.714958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1e598e165f64a6da5d538b0c0b2f334'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.715582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1e598e165f64a6da5d538b0c0b2f334'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.716184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1e598e165f64a6da5d538b0c0b2f334'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.724076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1e598e165f64a6da5d538b0c0b2f334'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:53.742714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1063/2120] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:53 2022] DELETE /identity/v3/users/c1e598e165f64a6da5d538b0c0b2f334 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:54.168439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:54.170036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:54.188930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d04627-246b-41bc-9840-cc5f0d6bdf3b admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:54.261763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:54.262433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:54.262705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:54.262977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] PATH_INFO: `/v3/projects/0a8e13da74b94aff8f92cb462613a7bb` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:54.264105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a8e13da74b94aff8f92cb462613a7bb)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:54.266553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:54.313387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a8e13da74b94aff8f92cb462613a7bb', 'request_id': 'req-8643ccf4-c953-46cc-bcb1-d4413baec575'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.321515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a8e13da74b94aff8f92cb462613a7bb', 'request_id': 'req-8643ccf4-c953-46cc-bcb1-d4413baec575'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.327418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a8e13da74b94aff8f92cb462613a7bb', 'request_id': 'req-8643ccf4-c953-46cc-bcb1-d4413baec575'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.339142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] The token cache is being invalidate because project 0a8e13da74b94aff8f92cb462613a7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:54.339430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.340067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8643ccf4-c953-46cc-bcb1-d4413baec575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.341653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1058/2121] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:54 2022] DELETE /identity/v3/projects/0a8e13da74b94aff8f92cb462613a7bb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:54.345621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:54.347110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:54.360186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384032d-0106-475b-b055-0ebaec8bc607 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:54.422439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:54.423104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:54.423778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:54.423778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] PATH_INFO: `/v3/users/09b7c920e6db4b0e96d132cc541b33b6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:54.424777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b7c920e6db4b0e96d132cc541b33b6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:54.426737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:54.474685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:03:54.477674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b7c920e6db4b0e96d132cc541b33b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.496011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b7c920e6db4b0e96d132cc541b33b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.496639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b7c920e6db4b0e96d132cc541b33b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.497299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b7c920e6db4b0e96d132cc541b33b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.506120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b7c920e6db4b0e96d132cc541b33b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:54.524327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1064/2122] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:03:54 2022] DELETE /identity/v3/users/09b7c920e6db4b0e96d132cc541b33b6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:03:54.598490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:54.598865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:54.599146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:54.620195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] MFA Rules not processed for user `8cdd0da2c7bc4139bcd48ef36a2ee183`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:03:54.621417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:54.690928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1059/2123] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:03:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:03:54.850162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:03:54.851628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:03:54.922608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74ed5b-0cb9-418d-8e2d-a3bcf43b318d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:03:54.929383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:03:54.929383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:03:54.929383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:03:54.929383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] PATH_INFO: `/v3/projects/f32d20be52124aa9beb62bbcf5e17f43` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:03:54.931867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f32d20be52124aa9beb62bbcf5e17f43)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:03:54.933782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:03:54.994695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f32d20be52124aa9beb62bbcf5e17f43', 'request_id': 'req-e57ed674-aa5a-4ac6-91c0-2f790e56236d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:55.001238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f32d20be52124aa9beb62bbcf5e17f43', 'request_id': 'req-e57ed674-aa5a-4ac6-91c0-2f790e56236d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:55.008863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f32d20be52124aa9beb62bbcf5e17f43', 'request_id': 'req-e57ed674-aa5a-4ac6-91c0-2f790e56236d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:55.021629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] The token cache is being invalidate because project f32d20be52124aa9beb62bbcf5e17f43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:03:55.021955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:55.022583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-e57ed674-aa5a-4ac6-91c0-2f790e56236d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:03:55.027327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1065/2124] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:03:54 2022] DELETE /identity/v3/projects/f32d20be52124aa9beb62bbcf5e17f43 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:04.682345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:05.018142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=11,keystone:SELECT=64,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:07.905243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:07.906691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:07.919892 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2c5ac-3752-49fc-923b-9197dd6c9543 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:07.978614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:07.979662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:07.979974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:07.980246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] PATH_INFO: `/v3/users/c3801be705a34f8aaafddcaa5c40b78c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:07.981784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3801be705a34f8aaafddcaa5c40b78c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:07.983760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.031809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3801be705a34f8aaafddcaa5c40b78c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.039683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3801be705a34f8aaafddcaa5c40b78c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.040809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3801be705a34f8aaafddcaa5c40b78c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.048087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3801be705a34f8aaafddcaa5c40b78c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.048750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3801be705a34f8aaafddcaa5c40b78c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.066334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1060/2125] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:04:07 2022] DELETE /identity/v3/users/c3801be705a34f8aaafddcaa5c40b78c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:08.070846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.071320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.071413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.090833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:08.092029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:08.148877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1066/2126] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:08.158549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.160267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:08.218204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c889361-4131-4f3f-aa34-c8b705007c57 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.222469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be6028-6459-437d-b356-0e001cef3c5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.223073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be6028-6459-437d-b356-0e001cef3c5a service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.223346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be6028-6459-437d-b356-0e001cef3c5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.223615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be6028-6459-437d-b356-0e001cef3c5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.232739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be6028-6459-437d-b356-0e001cef3c5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.234188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be6028-6459-437d-b356-0e001cef3c5a service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.244312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1061/2127] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:04:08.529705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.531141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:08.544151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe131db9-0101-4880-9b14-52fc25bda45c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.548771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.549401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.549699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.549960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] PATH_INFO: `/v3/projects/345d1b9ec923491b8e7f4d77a20a576c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.551171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=345d1b9ec923491b8e7f4d77a20a576c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.553053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.600254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345d1b9ec923491b8e7f4d77a20a576c', 'request_id': 'req-39f1148e-6292-4917-9d23-06d9f5193a38'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.606404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345d1b9ec923491b8e7f4d77a20a576c', 'request_id': 'req-39f1148e-6292-4917-9d23-06d9f5193a38'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.613739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345d1b9ec923491b8e7f4d77a20a576c', 'request_id': 'req-39f1148e-6292-4917-9d23-06d9f5193a38'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.624503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] The token cache is being invalidate because project 345d1b9ec923491b8e7f4d77a20a576c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:04:08.624793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.625920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-39f1148e-6292-4917-9d23-06d9f5193a38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:08.627431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1067/2128] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:04:08 2022] DELETE /identity/v3/projects/345d1b9ec923491b8e7f4d77a20a576c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:08.632569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.632858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.633140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.654219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:08.655618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:08.672546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:08.715768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1062/2129] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:08.723882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.724169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.724449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.743761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:08.744959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:08.753758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1068/2130] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:08.758254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.768292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d910f7-9df9-4220-a3de-2773ad650a89 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.773137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.773734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.774025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.774307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.774947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:04:08.775240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.776853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.789832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1063/2131] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:04:08.794179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.803558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581e28f-b65b-4222-aeef-1e63a3c104d3 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.808145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.808734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.809016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.809286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.809917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.811860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.839469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1069/2132] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:04:08 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:04:08.843543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.853351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189e05b-20ba-46d9-b983-120f789ed9dd admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.909470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.910091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.910363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.910633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.911284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.913192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.934893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1064/2133] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:04:08 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:04:08.939418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.949538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1d810-ff70-4ea1-b1e2-026b59d1df23 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.954409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.955002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.955278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.955546 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.956192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.957794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:08.967457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1070/2134] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:04:08.971253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:08.980928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f454e72-2fe5-4250-8f0b-50f733263e23 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:08.985913 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb6b3b2-1119-4753-964f-901ff2347a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:08.986521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6b3b2-1119-4753-964f-901ff2347a52 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:08.986798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6b3b2-1119-4753-964f-901ff2347a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:08.987064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb6b3b2-1119-4753-964f-901ff2347a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:08.987685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6b3b2-1119-4753-964f-901ff2347a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:08.989259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb6b3b2-1119-4753-964f-901ff2347a52 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:09.000081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1065/2135] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:04:09.004549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:09.015027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92efa094-815b-4471-b9fc-fa2ccb545317 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:09.020141 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:09.020783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:09.021061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:09.021336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] PATH_INFO: `/v3/projects/cd3d02b11bdf45fe99687fda3d1078cd/users/c8fe3c5c28964b768b2c0023fc62f163/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:09.036787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3d02b11bdf45fe99687fda3d1078cd, user_id=c8fe3c5c28964b768b2c0023fc62f163, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:09.039478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:09.053001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1071/2136] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:04:09 2022] PUT /identity/v3/projects/cd3d02b11bdf45fe99687fda3d1078cd/users/c8fe3c5c28964b768b2c0023fc62f163/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:09.057851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce568900-17f4-48b5-b2e2-03829d414734 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:09.058484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce568900-17f4-48b5-b2e2-03829d414734 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:09.058741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce568900-17f4-48b5-b2e2-03829d414734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:09.077920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ce568900-17f4-48b5-b2e2-03829d414734 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:09.079159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ce568900-17f4-48b5-b2e2-03829d414734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:09.136829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1066/2137] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:09.146722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:09.149456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:09.171488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:09.210232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d14fd9-bda1-49bd-aa3d-ad686644c2a1 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:09.214075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:09.214677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:09.214954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:09.215227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:09.224227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:09.225638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:09.235568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1072/2138] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:04:13.455992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.456461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.456557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.488639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] MFA Rules not processed for user `c8fe3c5c28964b768b2c0023fc62f163`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:13.489923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:13.508125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:13.556630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1067/2139] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:13.561701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:13.572763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7eff52-95e1-414b-9834-17dc3f010886 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:13.577341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:13.577935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.578247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.578544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.579330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:13.581185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:13.622918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1073/2140] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:04:13 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:04:13.627605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:13.644074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f70c5-7c31-44c6-ab09-36b83df41edb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:13.707613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:13.708325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.708677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.709018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.709760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:13.711837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:13.734362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1068/2141] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:04:13 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:04:13.739022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:13.749828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8baf4-3fcb-4dcf-b50a-5d3b1b893716 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:13.755210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:13.755973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.756247 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.756515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.757155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:13.758785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:13.770678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1074/2142] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:04:13.774919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:13.785894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6bdfc-cceb-4a04-b90e-ead8494bfad3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:13.791106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:13.791774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.792083 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.792358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] PATH_INFO: `/v3/projects/ebc2f3caf6f74c1192a858fc0f9aca5f/users/8baf517b712c4825ba0ac6c786d43c74/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.811084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc2f3caf6f74c1192a858fc0f9aca5f, user_id=8baf517b712c4825ba0ac6c786d43c74, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:13.813814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:13.828779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1069/2143] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:04:13 2022] PUT /identity/v3/projects/ebc2f3caf6f74c1192a858fc0f9aca5f/users/8baf517b712c4825ba0ac6c786d43c74/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:13.832820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:13.852991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f91d7-6f6b-4d8b-be90-bd1e6800d47c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:13.922079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:13.923294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.923575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.923847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.924497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:13.926142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:13.936679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1075/2144] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:04:13.940883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:13.951579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e26976-3b18-4e9a-9949-e6268fe7983d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:13.960048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc367d-63d8-4fa4-a388-d4d372f9ab5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:13.960720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc367d-63d8-4fa4-a388-d4d372f9ab5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:13.961007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc367d-63d8-4fa4-a388-d4d372f9ab5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:13.961261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc367d-63d8-4fa4-a388-d4d372f9ab5c admin admin] PATH_INFO: `/v3/projects/ebc2f3caf6f74c1192a858fc0f9aca5f/users/8baf517b712c4825ba0ac6c786d43c74/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:13.963283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc367d-63d8-4fa4-a388-d4d372f9ab5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc2f3caf6f74c1192a858fc0f9aca5f, user_id=8baf517b712c4825ba0ac6c786d43c74, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:13.965925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc367d-63d8-4fa4-a388-d4d372f9ab5c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:13.981511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1070/2145] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:04:13 2022] PUT /identity/v3/projects/ebc2f3caf6f74c1192a858fc0f9aca5f/users/8baf517b712c4825ba0ac6c786d43c74/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:15.963039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:15.965128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:15.981490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ed5d0-feb9-4f50-9988-392839778d14 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:16.053633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:16.055259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:16.055857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:16.056324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] PATH_INFO: `/v3/users/f312fd0cb75541099871236e6dd6ffac` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:16.058344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f312fd0cb75541099871236e6dd6ffac)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:16.060786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:16.103673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:16.164251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f312fd0cb75541099871236e6dd6ffac'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:16.176459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f312fd0cb75541099871236e6dd6ffac'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:16.177108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f312fd0cb75541099871236e6dd6ffac'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:16.183145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f312fd0cb75541099871236e6dd6ffac'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:16.199054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f312fd0cb75541099871236e6dd6ffac'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:16.267016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1076/2146] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:04:15 2022] DELETE /identity/v3/users/f312fd0cb75541099871236e6dd6ffac => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:16.272533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:16.273071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:16.273444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:16.317061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:16.318626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:16.457575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1071/2147] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:16.472810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:16.474511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:16.567591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870d3d0-fc36-4127-9a5b-fe9148f458b4 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:16.572222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d6207-2ba0-4241-90b2-1408a0b61c74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:16.573051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d6207-2ba0-4241-90b2-1408a0b61c74 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:16.573403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d6207-2ba0-4241-90b2-1408a0b61c74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:16.573765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d6207-2ba0-4241-90b2-1408a0b61c74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:16.586855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d6207-2ba0-4241-90b2-1408a0b61c74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:16.588715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d6207-2ba0-4241-90b2-1408a0b61c74 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:16.622298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1077/2148] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:04:17.255023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:17.255829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:17.279037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd23c8-5bcf-4665-a144-2838beded8eb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:17.294841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:17.295515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:17.295792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:17.296670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] PATH_INFO: `/v3/projects/cb7a4ece4b1c447dab3ba43e93dc7903` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:17.298422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7a4ece4b1c447dab3ba43e93dc7903)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:17.300346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:17.313968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:17.383182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7a4ece4b1c447dab3ba43e93dc7903', 'request_id': 'req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:17.397058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7a4ece4b1c447dab3ba43e93dc7903', 'request_id': 'req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:17.405919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7a4ece4b1c447dab3ba43e93dc7903', 'request_id': 'req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:17.428475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] The token cache is being invalidate because project cb7a4ece4b1c447dab3ba43e93dc7903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:04:17.429113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:17.430442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:17.447011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1072/2149] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:04:17 2022] DELETE /identity/v3/projects/cb7a4ece4b1c447dab3ba43e93dc7903 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:19.406891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:19.408213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:19.408502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:19.443257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] MFA Rules not processed for user `8baf517b712c4825ba0ac6c786d43c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:19.444491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:19.531891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1078/2150] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:19.546285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:19.548667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:19.639210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c98f1-83d9-476c-a8cb-4dd5ac8532cb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:19.645525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:19.646246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:19.655638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:19.655945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:19.657953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:19.721363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:19.725643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9ce80-b75c-4ffe-9ce7-27b3c8f81e27 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:19.738733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1073/2151] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:04:19.917895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:19.919375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:20.027015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:20.077417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932428-2b7c-48c8-82dc-adb15b2328aa None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:20.087947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:20.088649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:20.088930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:20.089203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] PATH_INFO: `/v3/users/a19a9a380ca74cceb3284f1b005af0de` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:20.090722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19a9a380ca74cceb3284f1b005af0de)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:20.092895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:20.188995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19a9a380ca74cceb3284f1b005af0de'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:20.204076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19a9a380ca74cceb3284f1b005af0de'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:20.204807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19a9a380ca74cceb3284f1b005af0de'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:20.205422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19a9a380ca74cceb3284f1b005af0de'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:20.215830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19a9a380ca74cceb3284f1b005af0de'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:20.245136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1079/2152] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:04:19 2022] DELETE /identity/v3/users/a19a9a380ca74cceb3284f1b005af0de => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:20.246662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:20.246957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:20.247236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:20.274343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:20.275501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:20.311622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:20.377695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1074/2153] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:20.384796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:20.386686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:20.492705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39e1b1-c995-4728-8852-ed72a75b5f3f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:20.497736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355910e-6e66-4f51-9c64-55bb43acdf96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:20.498490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355910e-6e66-4f51-9c64-55bb43acdf96 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:20.498934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355910e-6e66-4f51-9c64-55bb43acdf96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:20.499314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355910e-6e66-4f51-9c64-55bb43acdf96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:20.512029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355910e-6e66-4f51-9c64-55bb43acdf96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:20.513609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355910e-6e66-4f51-9c64-55bb43acdf96 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:20.526264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1080/2154] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:04:21.082792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.085397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:21.099874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b00c1-0ca4-4a11-ab6e-f0182fc4bf99 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.106126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.106996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.107439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.107878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] PATH_INFO: `/v3/projects/e2879f97af87438c8aacc1ffafba6a33` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.110727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2879f97af87438c8aacc1ffafba6a33)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.113335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.204650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2879f97af87438c8aacc1ffafba6a33', 'request_id': 'req-1e2037e5-6f07-4668-9622-53e9b5a32ee5'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:21.216535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2879f97af87438c8aacc1ffafba6a33', 'request_id': 'req-1e2037e5-6f07-4668-9622-53e9b5a32ee5'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:21.230249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2879f97af87438c8aacc1ffafba6a33', 'request_id': 'req-1e2037e5-6f07-4668-9622-53e9b5a32ee5'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:21.253350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] The token cache is being invalidate because project e2879f97af87438c8aacc1ffafba6a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:04:21.253646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:21.254342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1e2037e5-6f07-4668-9622-53e9b5a32ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:04:21.256153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1075/2155] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:04:21 2022] DELETE /identity/v3/projects/e2879f97af87438c8aacc1ffafba6a33 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:21.261741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.262049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.262325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.292524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:21.293920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:21.376474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:21.400104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1081/2156] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:21.402901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.403236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.403733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.432058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:21.433513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:21.450152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1076/2157] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:21.451189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.467973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7eb8d-6a45-4975-aaaf-535432733013 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.474694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.475589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.475872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.476140 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.476844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:04:21.477148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.478847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.501377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1082/2158] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:04:21.505488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.515657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178292ca-af9b-4c95-95b3-a2ac8ac75e64 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.525451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a184e3f-e497-45ee-b084-7160062b9bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.526127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a184e3f-e497-45ee-b084-7160062b9bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.526408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a184e3f-e497-45ee-b084-7160062b9bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.526679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a184e3f-e497-45ee-b084-7160062b9bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.527512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a184e3f-e497-45ee-b084-7160062b9bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.529431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a184e3f-e497-45ee-b084-7160062b9bf5 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.572480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1077/2159] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:04:21 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:04:21.586664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.598457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b7b0-b526-447a-b6d5-89da5b0fdf54 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.623950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.624678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.625728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.650595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] MFA Rules not processed for user `c8fe3c5c28964b768b2c0023fc62f163`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:21.652291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:21.685954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.686599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.686876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.687143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.688340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.690353 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.715146 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1083/2160] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:04:21 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:04:21.719129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.729670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1078/2161] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:04:21.730569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d92237-4b7f-4de8-9a48-f186575a49e7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.736383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.737068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.737269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.737538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.738215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.739814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.755126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1084/2162] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:04:21.759241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.774481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503712d-6828-4cad-9e5b-30f5b155a253 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.779632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78619f62-4a6a-4a47-b3d8-42190c98da31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.780502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619f62-4a6a-4a47-b3d8-42190c98da31 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.780790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619f62-4a6a-4a47-b3d8-42190c98da31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.781060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619f62-4a6a-4a47-b3d8-42190c98da31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.781753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78619f62-4a6a-4a47-b3d8-42190c98da31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.783763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78619f62-4a6a-4a47-b3d8-42190c98da31 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.795690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1079/2163] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:04:21.800345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.810786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d91aa6-7665-4df5-b15a-688bc3e31fde admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:21.818082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:21.819328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.819643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.819921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] PATH_INFO: `/v3/projects/0e942295e9b3478abd521c6f3e60e888/users/358ce5f5c35f415eb79322ae6d187b1d/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.837621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e942295e9b3478abd521c6f3e60e888, user_id=358ce5f5c35f415eb79322ae6d187b1d, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:21.840447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:21.862837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1085/2164] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:04:21 2022] PUT /identity/v3/projects/0e942295e9b3478abd521c6f3e60e888/users/358ce5f5c35f415eb79322ae6d187b1d/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:04:21.867755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afbaac-b4c5-4408-9d40-877fbadefcf0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:21.868038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afbaac-b4c5-4408-9d40-877fbadefcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:21.868309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69afbaac-b4c5-4408-9d40-877fbadefcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:21.889164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-69afbaac-b4c5-4408-9d40-877fbadefcf0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:21.890430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-69afbaac-b4c5-4408-9d40-877fbadefcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:21.906126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:21.961703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1080/2165] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:21.972117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:21.973803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:22.046345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5dc9-2e5a-406c-86a1-04486d577bdd admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:22.050600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:22.051349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:22.051726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:22.052101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:22.063762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:22.065504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:22.076536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1086/2166] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:04:22.791607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547245-4488-47a3-b354-76d63270e8b7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:22.792129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547245-4488-47a3-b354-76d63270e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:22.792229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db547245-4488-47a3-b354-76d63270e8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:22.821551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-db547245-4488-47a3-b354-76d63270e8b7 None None] MFA Rules not processed for user `358ce5f5c35f415eb79322ae6d187b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:22.822832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-db547245-4488-47a3-b354-76d63270e8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:22.904089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1081/2167] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:04:22.915235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:04:22.916801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:22.968757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:22.985594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83433ed0-490b-4867-bcea-bdf29e33b07c service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:04:22.989199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:04:22.989803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:22.990117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:22.990392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:22.999034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:04:23.001137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:04:23.011604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1087/2168] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:04:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:04:25.462028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:04:25.462647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:04:25.462647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:04:25.483032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] MFA Rules not processed for user `358ce5f5c35f415eb79322ae6d187b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:04:25.484366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:04:25.492468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1082/2169] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:04:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:04:33.006656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=8 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:04:35.477401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=53 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:05:07.528086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:07.538692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329114e-f5da-48ed-a301-ae5e122e2c48 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:07.542223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:07.542836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:07.543109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:07.543372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:07.545038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:07.606494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:07.607931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:07.617450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1088/2170] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:17.613108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=30 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:05:39.816774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:39.827520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ba3b7-9156-4f08-9ee2-ecdeacab9453 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:39.831191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:39.831799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:39.832074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:39.832343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:39.834183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:39.908216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:39.909560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:39.921968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1083/2171] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:39.987530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:39.996982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee843550-5b83-4976-aede-22902957c0c8 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:40.000658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:40.001346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:40.001625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:40.001897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:40.003639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:40.017224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:40.018688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76910301-f552-42c7-8b41-bec2d2986751 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:40.030299 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1089/2172] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:49.917018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=34 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:05:50.025332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=6 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:05:50.981195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:50.990673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212109c5-6d41-4a88-ab2d-2ab767ef3640 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:50.994422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:50.995024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:50.995302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:50.995577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:50.997441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:51.056368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:51.057840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:51.066979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1084/2173] 10.222.0.50 () {60 vars in 1396 bytes} [Sat Sep 24 05:05:50 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:52.082406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233bcb-5573-4210-94a5-493d800f021c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:52.082954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233bcb-5573-4210-94a5-493d800f021c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:52.083057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97233bcb-5573-4210-94a5-493d800f021c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:52.102754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-97233bcb-5573-4210-94a5-493d800f021c None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:05:52.103948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-97233bcb-5573-4210-94a5-493d800f021c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:52.160027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1090/2174] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:05:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:05:52.204053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:52.205499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:52.277447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29782c0b-a2ca-4f29-97d6-150fadb6b699 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:52.282799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d77da-836f-41df-8a31-770423025865 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:52.283402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d77da-836f-41df-8a31-770423025865 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:52.283675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d77da-836f-41df-8a31-770423025865 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:52.283944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d77da-836f-41df-8a31-770423025865 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:52.292816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d77da-836f-41df-8a31-770423025865 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:52.294444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d77da-836f-41df-8a31-770423025865 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:52.306604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1085/2175] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:05:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:05:54.332525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97233bcb-5573-4210-94a5-493d800f021c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:54.334567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-97233bcb-5573-4210-94a5-493d800f021c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:54.350929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97233bcb-5573-4210-94a5-493d800f021c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:54.356413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:54.357013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:54.357295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:54.357582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:54.359464 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:54.423355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:54.425096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:54.436953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1091/2176] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:57.696775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:57.697676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:57.697676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:57.723087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] MFA Rules not processed for user `358ce5f5c35f415eb79322ae6d187b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:05:57.725100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:57.732748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1086/2177] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:05:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:05:57.985787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:57.997267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d0003-97b0-49c8-a058-73465a637a11 service nova] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:58.001625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:58.002268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:58.002546 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:58.002816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:58.004600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:58.015495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:58.016972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:58.026198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1092/2178] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:58.372451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:58.374784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:58.487148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa2ff2-3b34-48da-adb1-363687ca39b4 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:58.493457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:58.494192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:58.494537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:58.494879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:58.496788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:58.510553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:58.512238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:58.523595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1087/2179] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:05:59.388696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:05:59.390518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:59.403592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb18e5-8cf1-4f88-94d6-b006760f6012 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:05:59.408446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:05:59.409342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:05:59.409929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:05:59.410463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:05:59.413089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:05:59.616103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:05:59.617786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21dd89c-cd49-4710-a153-0a6281a7f14d service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:05:59.650901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1093/2180] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:05:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:06.521796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:06.523626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:06.535071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ae5dc-c72e-4ab6-a997-4822e24a0aed service placement] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:06.539283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:06.539875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:06.540151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:06.540420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:06.542247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:06.551804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:06.553221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eaaf3-a8e4-4f92-b93b-a9c574a607e3 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:06.556260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:06:06.562112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1088/2181] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:09.624478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=87 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:06:13.643237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134d593-c2a7-4a9d-86da-7367bfcd561e None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:13.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134d593-c2a7-4a9d-86da-7367bfcd561e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:13.643857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134d593-c2a7-4a9d-86da-7367bfcd561e None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:13.645127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1094/2182] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:06:13.653904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6a645-0dbf-4ba7-b1be-c365db1738ee None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:13.654315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6a645-0dbf-4ba7-b1be-c365db1738ee None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:13.654315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6a645-0dbf-4ba7-b1be-c365db1738ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:13.677966 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1dc6a645-0dbf-4ba7-b1be-c365db1738ee None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:06:13.679371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1dc6a645-0dbf-4ba7-b1be-c365db1738ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:13.688631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1089/2183] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:06:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:06:13.701201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134d593-c2a7-4a9d-86da-7367bfcd561e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:13.702928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5134d593-c2a7-4a9d-86da-7367bfcd561e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:13.764195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134d593-c2a7-4a9d-86da-7367bfcd561e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:13.767756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83420f15-2585-4fa7-af4a-2094746e1879 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:13.768365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420f15-2585-4fa7-af4a-2094746e1879 service glance] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:13.768667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420f15-2585-4fa7-af4a-2094746e1879 service glance] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:13.768936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83420f15-2585-4fa7-af4a-2094746e1879 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:13.779815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83420f15-2585-4fa7-af4a-2094746e1879 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:13.781199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83420f15-2585-4fa7-af4a-2094746e1879 service glance] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:13.792520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1095/2184] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:13.875212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633723ff-7001-4870-b6d1-f4f1bbc8d777 None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:13.875497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633723ff-7001-4870-b6d1-f4f1bbc8d777 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:13.875775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633723ff-7001-4870-b6d1-f4f1bbc8d777 None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:13.881660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1090/2185] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:06:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:06:13.887147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ce130-5852-4d99-a104-300561ab6c33 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:13.887702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ce130-5852-4d99-a104-300561ab6c33 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:13.887979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ce130-5852-4d99-a104-300561ab6c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:13.909166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2c5ce130-5852-4d99-a104-300561ab6c33 None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:06:13.910416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2c5ce130-5852-4d99-a104-300561ab6c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:13.920065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1096/2186] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:06:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:06:13.932482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633723ff-7001-4870-b6d1-f4f1bbc8d777 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:13.947978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633723ff-7001-4870-b6d1-f4f1bbc8d777 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:13.951644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b3f3d4-3821-4e84-81bb-f682d0e5ea06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:13.952252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3f3d4-3821-4e84-81bb-f682d0e5ea06 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:13.952529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3f3d4-3821-4e84-81bb-f682d0e5ea06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:13.952799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b3f3d4-3821-4e84-81bb-f682d0e5ea06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:13.960192 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3f3d4-3821-4e84-81bb-f682d0e5ea06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:13.962197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b3f3d4-3821-4e84-81bb-f682d0e5ea06 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:13.970396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1091/2187] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:14.121733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f77a1b-a1ed-44b2-abb6-5aedf3e68b80 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:14.122143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f77a1b-a1ed-44b2-abb6-5aedf3e68b80 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:14.122507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f77a1b-a1ed-44b2-abb6-5aedf3e68b80 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:14.123843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1097/2188] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:06:14.129426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad5255-92b3-4f8c-ba95-c10303780868 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:14.129783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad5255-92b3-4f8c-ba95-c10303780868 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:14.130157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad5255-92b3-4f8c-ba95-c10303780868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:14.169310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7cad5255-92b3-4f8c-ba95-c10303780868 None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:06:14.170555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7cad5255-92b3-4f8c-ba95-c10303780868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:14.180082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1092/2189] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:06:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:06:14.190746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f77a1b-a1ed-44b2-abb6-5aedf3e68b80 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:14.203242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f77a1b-a1ed-44b2-abb6-5aedf3e68b80 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:14.206793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:14.207453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:14.207816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:14.208204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:14.220257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:14.221771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:14.233575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1098/2190] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:17.653690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:17.654581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:17.655008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:17.684423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] MFA Rules not processed for user `74a8cc19ab204424ab23c62915ac2bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:06:17.685632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:17.692813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1093/2191] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:06:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:06:18.000327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:18.009508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937fd9f-a3c0-4867-a31d-966cc554e4b1 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:18.013422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:18.014038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:18.014320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:18.014586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:18.016330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:18.025841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:18.027284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919a6259-a0b0-4e81-90a2-0deaca0fd8e0 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:18.034843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1099/2192] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:21.077554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:21.094176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079cc74-541e-4982-867c-a71b4ce1e42d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:21.101457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:21.102276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:21.102677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:21.103057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:21.104909 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:21.119640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:21.121534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:21.133533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1094/2193] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:28.031534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=41 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:06:31.128128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=22 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:06:41.490600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:41.490600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:41.490600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:41.515279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:06:41.516618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:41.524628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1100/2194] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:06:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:06:41.538469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:41.549545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340483f6-6842-4a3b-827e-4763bb4b3c25 service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:41.554581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:41.555178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:41.555456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:41.555728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:41.563788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:41.565353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:41.574131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1095/2195] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:06:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:06:43.391912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:06:43.425130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271e602-3732-4d81-8b15-3b22dfc37519 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:06:43.445729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:06:43.446394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:06:43.446665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:06:43.447119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:06:43.449269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:06:43.467543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:06:43.469300 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4be61-0022-483f-b1c6-766eaf71e93b service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:06:43.481180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1101/2196] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:06:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:06:51.570768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=6 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:06:53.475834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=9 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:12.064580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:12.066435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:12.082360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb460-8cde-4560-b49c-4dac889dca91 service swift] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:12.088526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:12.089126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:12.089403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:12.089669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:12.091388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:12.103635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:12.105229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:12.115693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1096/2197] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:07:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:07:18.833519 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:18.834027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:18.834135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:18.852683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:18.854418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:18.861424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1102/2198] 10.222.0.50 () {58 vars in 967 bytes} [Sat Sep 24 05:07:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:07:18.886434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:18.898194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543421-3fb0-4e37-b2fd-fd0a22f1fc62 service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:18.903593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:18.904197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:18.904480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:18.904759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:18.913128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:18.914745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:18.923634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1097/2199] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:07:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:07:26.117100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:26.130126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2efa463-b569-4746-9c02-c867c3e372cb None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:26.136125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:26.136714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:26.136989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:26.137264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:26.139160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:26.195589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:26.197749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:26.206882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1103/2200] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:07:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:07:28.920189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=12 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:36.202452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=31 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:37.835105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:37.845912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ebf252-bcb8-498c-a125-1fb095542ee7 service swift] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:37.850964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:37.851563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:37.851844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:37.852119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:37.854369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:37.865604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:37.867205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:37.878105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1098/2201] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:07:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:07:37.897726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:37.909978 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e2b12-c778-45c4-a888-94a2f68e35ce service nova] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:37.915269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:37.916676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:37.917145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:37.917877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] PATH_INFO: `/v3/users/358ce5f5c35f415eb79322ae6d187b1d` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:37.919439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] RBAC: Authorizing `identity:delete_user(user_id=358ce5f5c35f415eb79322ae6d187b1d)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:37.921461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:37.973751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '358ce5f5c35f415eb79322ae6d187b1d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:37.982789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '358ce5f5c35f415eb79322ae6d187b1d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:37.983398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '358ce5f5c35f415eb79322ae6d187b1d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:37.984505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '358ce5f5c35f415eb79322ae6d187b1d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:37.992456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-dedd2038-7700-4133-accb-704ecd8b277f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '358ce5f5c35f415eb79322ae6d187b1d'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:38.014574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1104/2202] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:07:37 2022] DELETE /identity/v3/users/358ce5f5c35f415eb79322ae6d187b1d => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:07:38.193411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.204186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce3508-b464-424c-97c6-ba01ea5bb66a service placement] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.261544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.262331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.262609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.262880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] PATH_INFO: `/v3/projects/0e942295e9b3478abd521c6f3e60e888` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.263981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e942295e9b3478abd521c6f3e60e888)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.265865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.311294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e942295e9b3478abd521c6f3e60e888', 'request_id': 'req-656aa3ef-d096-4805-a87f-d9f5df1de14a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:38.317955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e942295e9b3478abd521c6f3e60e888', 'request_id': 'req-656aa3ef-d096-4805-a87f-d9f5df1de14a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:38.324210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e942295e9b3478abd521c6f3e60e888', 'request_id': 'req-656aa3ef-d096-4805-a87f-d9f5df1de14a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:38.334899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] The token cache is being invalidate because project 0e942295e9b3478abd521c6f3e60e888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:07:38.335186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:38.335825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-656aa3ef-d096-4805-a87f-d9f5df1de14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:07:38.337264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1099/2203] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:07:38 2022] DELETE /identity/v3/projects/0e942295e9b3478abd521c6f3e60e888 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:07:38.342979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.343266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.343556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.362687 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:38.363881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:38.421400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1105/2204] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:07:38.430193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.430474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.430766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.450846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:38.452061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:38.459737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1100/2205] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:07:38.464340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.473846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c9ce53-86eb-48a9-86ac-047154083bdd None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.478512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.479103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.479380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.479652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.480292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:07:38.480591 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.482203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.493393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1106/2206] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:07:38.497385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.506403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357ebc1-de8b-4613-a435-dddd50934429 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.511517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.512155 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.512361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.512717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.513243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.515156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.542661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1101/2207] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:07:38 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:07:38.546885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.556607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e43cd-2f88-4507-9abc-73105a44124c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.610600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.611125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.611399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.611731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.612292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.614223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.622058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:38.636214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1107/2208] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:07:38 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:07:38.640386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.651256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca34615b-70a1-4f1f-8696-c457f8db2fce admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.656147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.656751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.657027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.657299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.657969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.659620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.669724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1102/2209] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:07:38.673638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.683968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc309f21-67ab-4cb9-a11c-7e93c09014b7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.688491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a244f0-5655-45c7-b348-5a11be1f02b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.689078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a244f0-5655-45c7-b348-5a11be1f02b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.689354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a244f0-5655-45c7-b348-5a11be1f02b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.689621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a244f0-5655-45c7-b348-5a11be1f02b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.690312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a244f0-5655-45c7-b348-5a11be1f02b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.691877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a244f0-5655-45c7-b348-5a11be1f02b2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.702626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1108/2210] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:07:38.706495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.716163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9edc3-de80-4f24-9be2-23bbcef93af8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.720729 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.721371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.721650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.722050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] PATH_INFO: `/v3/projects/fef19544ec414094953daea6d545a361/users/60edcebcccaa4da99ffddadd3e27c1c0/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.736451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef19544ec414094953daea6d545a361, user_id=60edcebcccaa4da99ffddadd3e27c1c0, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.739137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.751945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1103/2211] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:07:38 2022] PUT /identity/v3/projects/fef19544ec414094953daea6d545a361/users/60edcebcccaa4da99ffddadd3e27c1c0/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:07:38.756929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a97e1-4808-45ad-85e2-145e6792eafc None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.757229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a97e1-4808-45ad-85e2-145e6792eafc None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.757506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a97e1-4808-45ad-85e2-145e6792eafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.776667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-945a97e1-4808-45ad-85e2-145e6792eafc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:38.778188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-945a97e1-4808-45ad-85e2-145e6792eafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:38.835967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1109/2212] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:07:38.845720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:38.847481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:38.861521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:38.906609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ad621b-688d-4236-b497-df2b67fe95a0 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:38.910272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:38.910875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:38.911153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:38.911426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:38.920195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:38.921619 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:38.930859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1104/2213] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:07:42.878333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:42.879157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:42.879612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:42.915118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] MFA Rules not processed for user `60edcebcccaa4da99ffddadd3e27c1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:42.916368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:42.984848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1110/2214] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:07:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:07:42.989547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:43.000333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee1864-beda-462d-bf77-df561dee2120 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:43.019667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:43.020571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:43.020854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:43.022107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:43.022825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:43.024772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:43.056044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1105/2215] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:07:42 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:07:43.060046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:43.073833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68896bf-2c37-4ce3-b6c5-f7a6fb18b911 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:43.136675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:43.137575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:43.138041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:43.138518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:43.139174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:43.141094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:43.159699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:43.166036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1111/2216] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:07:43 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:07:43.170382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:43.181382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce2850-8d6b-4915-a021-3efe5d2921ea admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:43.186956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:43.187581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:43.187861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:43.188133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:43.188773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:43.190406 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:43.202476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1106/2217] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:07:43.206580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:43.218153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ee2b8-1d59-44f5-9265-98589118189b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:43.223130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:43.223821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:43.224082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:43.224352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] PATH_INFO: `/v3/projects/edba865cd0f14c16ba0060b3c031cefa/users/e0b2e05dd2694583b70c8914519ac343/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:43.240700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edba865cd0f14c16ba0060b3c031cefa, user_id=e0b2e05dd2694583b70c8914519ac343, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:43.243393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:43.257393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1112/2218] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:07:43 2022] PUT /identity/v3/projects/edba865cd0f14c16ba0060b3c031cefa/users/e0b2e05dd2694583b70c8914519ac343/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:07:43.261711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:43.272009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9ea54-1376-4ccb-9746-7f453b5e4015 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:43.331267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a501cc-d94e-4d06-9bc1-9fa4873023a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:43.331965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a501cc-d94e-4d06-9bc1-9fa4873023a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:43.332331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a501cc-d94e-4d06-9bc1-9fa4873023a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:43.332660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a501cc-d94e-4d06-9bc1-9fa4873023a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:43.333378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a501cc-d94e-4d06-9bc1-9fa4873023a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:43.335111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a501cc-d94e-4d06-9bc1-9fa4873023a0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:43.348422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1107/2219] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:07:43.350465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:43.361644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67d0271-e6d6-46d4-9b3e-4e7641c7dc58 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:43.366474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:43.367137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:43.367413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:43.367685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] PATH_INFO: `/v3/projects/edba865cd0f14c16ba0060b3c031cefa/users/e0b2e05dd2694583b70c8914519ac343/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:43.369668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=edba865cd0f14c16ba0060b3c031cefa, user_id=e0b2e05dd2694583b70c8914519ac343, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:43.372390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:43.383333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1113/2220] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:07:43 2022] PUT /identity/v3/projects/edba865cd0f14c16ba0060b3c031cefa/users/e0b2e05dd2694583b70c8914519ac343/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:07:48.018559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa997d-23a7-4c69-beba-831deec78681 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:48.019030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa997d-23a7-4c69-beba-831deec78681 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:48.020103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa997d-23a7-4c69-beba-831deec78681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:48.052752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-feaa997d-23a7-4c69-beba-831deec78681 None None] MFA Rules not processed for user `e0b2e05dd2694583b70c8914519ac343`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:48.054375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-feaa997d-23a7-4c69-beba-831deec78681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:48.170613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1108/2221] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:07:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:07:48.185616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:07:48.187552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:48.276614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df0e8c0-58ad-482b-a1fc-3badd51ab13f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:07:48.282489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d02f-8222-47d0-86e5-67b12082b3a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:07:48.283094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d02f-8222-47d0-86e5-67b12082b3a6 service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:48.283367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d02f-8222-47d0-86e5-67b12082b3a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:48.283631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d02f-8222-47d0-86e5-67b12082b3a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:48.344388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6d02f-8222-47d0-86e5-67b12082b3a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:07:48.346022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6d02f-8222-47d0-86e5-67b12082b3a6 service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:07:48.356010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1114/2222] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:07:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:07:50.546216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:07:50.546741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:07:50.548266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:07:50.559427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:07:50.575452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] MFA Rules not processed for user `8cdd0da2c7bc4139bcd48ef36a2ee183`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:07:50.576396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:07:50.642440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1109/2223] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:07:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:07:58.351256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=72,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:08:00.634683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=23 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:08:15.418326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a5032-ca13-401e-b679-4ed6fffeb4b9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:15.418834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a5032-ca13-401e-b679-4ed6fffeb4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:15.418941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a5032-ca13-401e-b679-4ed6fffeb4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:15.438893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-ea9a5032-ca13-401e-b679-4ed6fffeb4b9 None None] MFA Rules not processed for user `60edcebcccaa4da99ffddadd3e27c1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:08:15.440291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ea9a5032-ca13-401e-b679-4ed6fffeb4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:08:15.447208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1115/2224] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:08:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:08:25.432546 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:08:35.658993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:08:35.660403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:08:35.715920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761b0bc-8ca9-49d6-9250-46fc435f625d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:08:35.719979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dfd9ba-b65a-40c6-aa24-b8832cd1c9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:08:35.720575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd9ba-b65a-40c6-aa24-b8832cd1c9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:35.720846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd9ba-b65a-40c6-aa24-b8832cd1c9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:35.721114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd9ba-b65a-40c6-aa24-b8832cd1c9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:35.729814 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfd9ba-b65a-40c6-aa24-b8832cd1c9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:08:35.731253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dfd9ba-b65a-40c6-aa24-b8832cd1c9c8 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:08:35.741291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1110/2225] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:08:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:08:45.736880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=26 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:08:46.578123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b035cba-1b28-4693-b083-f57ec3299113 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:46.578727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b035cba-1b28-4693-b083-f57ec3299113 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:46.579795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b035cba-1b28-4693-b083-f57ec3299113 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:46.581282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1116/2226] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:08:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:08:46.587533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda2368-1dd8-4b68-bb8d-595b51a9af0a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:46.587970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda2368-1dd8-4b68-bb8d-595b51a9af0a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:46.588337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda2368-1dd8-4b68-bb8d-595b51a9af0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:46.640415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-fcda2368-1dd8-4b68-bb8d-595b51a9af0a None None] MFA Rules not processed for user `266e3f374e794afca24caf914399a7d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:08:46.641816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fcda2368-1dd8-4b68-bb8d-595b51a9af0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:08:46.670313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1111/2227] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:08:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:08:46.684791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b035cba-1b28-4693-b083-f57ec3299113 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:08:46.696840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b035cba-1b28-4693-b083-f57ec3299113 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:08:46.700669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3b5950-e738-4ba7-a1b9-cf143b05eeae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:08:46.701387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5950-e738-4ba7-a1b9-cf143b05eeae service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:46.701729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5950-e738-4ba7-a1b9-cf143b05eeae service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:46.702113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3b5950-e738-4ba7-a1b9-cf143b05eeae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:46.711002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5950-e738-4ba7-a1b9-cf143b05eeae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:08:46.713520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3b5950-e738-4ba7-a1b9-cf143b05eeae service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:08:46.725532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1117/2228] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:08:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:08:47.854496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c554bd-155a-475d-9870-898513c4e42f None None] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:47.855014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c554bd-155a-475d-9870-898513c4e42f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:47.855108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c554bd-155a-475d-9870-898513c4e42f None None] PATH_INFO: `/` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:47.856390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1112/2229] 10.222.0.50 () {54 vars in 921 bytes} [Sat Sep 24 05:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:08:47.861731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9cf3a-57cd-43d9-9f4f-04ed3b4fcaca None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:47.862185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9cf3a-57cd-43d9-9f4f-04ed3b4fcaca None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:47.862294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9cf3a-57cd-43d9-9f4f-04ed3b4fcaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:47.894383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b3d9cf3a-57cd-43d9-9f4f-04ed3b4fcaca None None] MFA Rules not processed for user `2451d53fb9974222827b6520b8bce5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:08:47.895579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b3d9cf3a-57cd-43d9-9f4f-04ed3b4fcaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:08:47.947696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1118/2230] 10.222.0.50 () {58 vars in 1018 bytes} [Sat Sep 24 05:08:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:08:47.959050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c554bd-155a-475d-9870-898513c4e42f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:08:47.969873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c554bd-155a-475d-9870-898513c4e42f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:08:47.973577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf64c4-48b9-415e-8afb-639a224dbf9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:08:47.974242 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf64c4-48b9-415e-8afb-639a224dbf9d service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:47.974520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf64c4-48b9-415e-8afb-639a224dbf9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:47.974792 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf64c4-48b9-415e-8afb-639a224dbf9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:47.985636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf64c4-48b9-415e-8afb-639a224dbf9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:08:47.987141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf64c4-48b9-415e-8afb-639a224dbf9d service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:08:47.997825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1113/2231] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:08:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:08:51.842504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0b34-f87a-4be7-b43f-a109c7995446 None None] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:51.843539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0b34-f87a-4be7-b43f-a109c7995446 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:51.843811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d0b34-f87a-4be7-b43f-a109c7995446 None None] PATH_INFO: `/` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:51.845159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1119/2232] 10.222.0.50 () {54 vars in 961 bytes} [Sat Sep 24 05:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 24 05:08:51.850454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c42f-fe9e-45ab-9392-5e54bc7aff3d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:51.850949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c42f-fe9e-45ab-9392-5e54bc7aff3d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:51.851068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699c42f-fe9e-45ab-9392-5e54bc7aff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:51.902553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-5699c42f-fe9e-45ab-9392-5e54bc7aff3d None None] MFA Rules not processed for user `fdf673c05e4443e8bce9b9d7371d1fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:08:51.903699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5699c42f-fe9e-45ab-9392-5e54bc7aff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:08:51.974052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1114/2233] 10.222.0.50 () {58 vars in 1058 bytes} [Sat Sep 24 05:08:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:08:51.981664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d0b34-f87a-4be7-b43f-a109c7995446 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:08:51.994849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d0b34-f87a-4be7-b43f-a109c7995446 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:08:51.999812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:08:52.000395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:08:52.000673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:08:52.000999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:08:52.002718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:08:52.013940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:08:52.016159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e16d0-e7ac-4434-93ab-908558dae023 service placement] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:08:52.025698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1120/2234] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:08:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:09:00.909191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:00.909781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:00.909781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:00.930030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] MFA Rules not processed for user `0948754217fc4ff2ba0241fa35feaf90`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:00.931249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:00.983783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1115/2235] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:02.021660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=36 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:10.977398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:18.782308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:18.782784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:18.782980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:18.801907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] MFA Rules not processed for user `74a8cc19ab204424ab23c62915ac2bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:18.803139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:18.856402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1121/2236] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:28.849786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=25 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:34.667299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:34.687612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6ac2-9170-4144-9a35-21fc2a05da7f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:34.691303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:34.691901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:34.692176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:34.692441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:34.694233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:34.779998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:34.781437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:34.800370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1116/2237] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:09:39.298688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:39.300148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:39.316628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a9f66-18b9-4381-9a9c-24e8730788f4 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:39.323354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:39.324008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:39.325541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:39.330480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:39.333033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:39.347598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:39.349287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:39.366985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1122/2238] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:09:39.368125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:39.369661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:39.392612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0531c-ce23-4398-855c-cdcc774dc065 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:39.397488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:39.398151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:39.398549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:39.398942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] PATH_INFO: `/v3/users/8cdd0da2c7bc4139bcd48ef36a2ee183` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:39.400005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cdd0da2c7bc4139bcd48ef36a2ee183)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:39.402141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:39.449831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cdd0da2c7bc4139bcd48ef36a2ee183'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.456880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cdd0da2c7bc4139bcd48ef36a2ee183'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.457505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cdd0da2c7bc4139bcd48ef36a2ee183'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.464590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cdd0da2c7bc4139bcd48ef36a2ee183'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.465207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-bceb37ee-7e58-4686-9a59-825b68c098ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cdd0da2c7bc4139bcd48ef36a2ee183'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.482602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1117/2239] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:09:39 2022] DELETE /identity/v3/users/8cdd0da2c7bc4139bcd48ef36a2ee183 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:09:39.654212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:39.668382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01a7ca-4a12-49fc-ba7b-3b78e3dc14d2 service placement] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:39.735955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:39.736539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:39.736880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:39.737186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] PATH_INFO: `/v3/projects/a18a3756ba2b460d977f738a115ebfe0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:39.738390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18a3756ba2b460d977f738a115ebfe0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:39.740387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:39.788248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18a3756ba2b460d977f738a115ebfe0', 'request_id': 'req-ea962295-38cb-4b75-ad02-f7535c992f31'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.793710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18a3756ba2b460d977f738a115ebfe0', 'request_id': 'req-ea962295-38cb-4b75-ad02-f7535c992f31'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.800724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18a3756ba2b460d977f738a115ebfe0', 'request_id': 'req-ea962295-38cb-4b75-ad02-f7535c992f31'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.811696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] The token cache is being invalidate because project a18a3756ba2b460d977f738a115ebfe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:09:39.812003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.812649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea962295-38cb-4b75-ad02-f7535c992f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:39.814103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1123/2240] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:09:39 2022] DELETE /identity/v3/projects/a18a3756ba2b460d977f738a115ebfe0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:39.819249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:39.819535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:39.819808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:39.864031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:39.868286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:39.870246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:39.928309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1118/2241] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:39.937096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:39.937374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:39.937641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:39.958125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:39.959331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:39.967042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1124/2242] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:39.971761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:39.981335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49314ff0-ea03-4ca2-ba64-0ebd00fd118b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:39.985854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:39.986547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:39.986790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:39.987065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:39.987706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:09:39.988004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:39.989587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.000934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1119/2243] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:09:40.005553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:40.014613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea28bfa-aac2-444a-be48-2b48ed2d9cca None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:40.019722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:40.020312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.020584 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.020848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.021469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:40.023377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.049545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1125/2244] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:09:40 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:40.053651 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:40.063703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2430654a-fe09-44a6-9f1a-dcdd1bf1c64c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:40.118505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:40.119109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.119384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.119652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.120281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:40.122200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.144741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1120/2245] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:09:40 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:40.148933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:40.160849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea4452-8554-4a6a-84e8-e774c1339739 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:40.165481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:40.166100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.166379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.166678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.167308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:40.168913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.178587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1126/2246] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:40.182459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:40.192773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1ee28-b8e4-428b-824d-5f753d4ad779 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:40.197329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fb208-57d1-4c9a-8349-2c3c2cd04bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:40.197915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb208-57d1-4c9a-8349-2c3c2cd04bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.198218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb208-57d1-4c9a-8349-2c3c2cd04bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.198486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fb208-57d1-4c9a-8349-2c3c2cd04bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.199117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb208-57d1-4c9a-8349-2c3c2cd04bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:40.200682 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fb208-57d1-4c9a-8349-2c3c2cd04bb1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.211533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1121/2247] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:40.215400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:40.225052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ba0cd-ef9e-4aa7-8757-816625104e0c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:40.229594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:40.230256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.230531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.230799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] PATH_INFO: `/v3/projects/c07d1da8ffb14033975b604382936e95/users/ce2f64f4f6a64d5985f7a9126a2941d8/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.244912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07d1da8ffb14033975b604382936e95, user_id=ce2f64f4f6a64d5985f7a9126a2941d8, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:40.247489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.260388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1127/2248] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:40 2022] PUT /identity/v3/projects/c07d1da8ffb14033975b604382936e95/users/ce2f64f4f6a64d5985f7a9126a2941d8/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:40.270867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.271151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.271421 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.298927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] MFA Rules not processed for user `ce2f64f4f6a64d5985f7a9126a2941d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:40.300165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:40.363030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1122/2249] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:40.373721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:40.375507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:40.393684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:40.455071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357bf299-6223-45d0-a7ce-48bebb77384f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:40.458770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a828058-7245-48be-b9c2-18eb7f3f3db7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:40.459367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a828058-7245-48be-b9c2-18eb7f3f3db7 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:40.459646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a828058-7245-48be-b9c2-18eb7f3f3db7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:40.459918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a828058-7245-48be-b9c2-18eb7f3f3db7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:40.468056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a828058-7245-48be-b9c2-18eb7f3f3db7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:40.469468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a828058-7245-48be-b9c2-18eb7f3f3db7 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:40.478167 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1128/2250] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:09:49.150809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:49.154974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:49.214020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d82055-ff56-4798-86a0-234e5bed6711 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:49.218479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:49.219110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:49.219388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:49.219663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] PATH_INFO: `/v3/users/ce2f64f4f6a64d5985f7a9126a2941d8` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:49.220776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2f64f4f6a64d5985f7a9126a2941d8)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:49.223217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:49.269019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2f64f4f6a64d5985f7a9126a2941d8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.275478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2f64f4f6a64d5985f7a9126a2941d8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.276112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2f64f4f6a64d5985f7a9126a2941d8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.283269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2f64f4f6a64d5985f7a9126a2941d8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.284420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2f64f4f6a64d5985f7a9126a2941d8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.302085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1123/2251] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:09:49 2022] DELETE /identity/v3/users/ce2f64f4f6a64d5985f7a9126a2941d8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:49.306643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:49.307126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:49.307225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:49.325709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:49.326940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:49.383843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1129/2252] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:49.394166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:49.395520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:49.477884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd935ee4-03ec-496d-8114-2fbf559423b9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:49.483259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e3d42-e6ab-4fe9-87af-36c97688f212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:49.483872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e3d42-e6ab-4fe9-87af-36c97688f212 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:49.484149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e3d42-e6ab-4fe9-87af-36c97688f212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:49.484424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e3d42-e6ab-4fe9-87af-36c97688f212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:49.495287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e3d42-e6ab-4fe9-87af-36c97688f212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:49.499196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e3d42-e6ab-4fe9-87af-36c97688f212 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:49.510815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1124/2253] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:09:49.773324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:49.775485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:49.792529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d56ac5-f7de-406d-8d7c-be0fde8a2588 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:49.798379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:49.799139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:49.799310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:49.799716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] PATH_INFO: `/v3/projects/c07d1da8ffb14033975b604382936e95` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:49.800680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07d1da8ffb14033975b604382936e95)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:49.802656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:49.850188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07d1da8ffb14033975b604382936e95', 'request_id': 'req-ee71cbca-8c50-4172-859b-4fac63d125a8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.855658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07d1da8ffb14033975b604382936e95', 'request_id': 'req-ee71cbca-8c50-4172-859b-4fac63d125a8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.862462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07d1da8ffb14033975b604382936e95', 'request_id': 'req-ee71cbca-8c50-4172-859b-4fac63d125a8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.873097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] The token cache is being invalidate because project c07d1da8ffb14033975b604382936e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:09:49.873381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.874003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee71cbca-8c50-4172-859b-4fac63d125a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:49.875455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1130/2254] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:09:49 2022] DELETE /identity/v3/projects/c07d1da8ffb14033975b604382936e95 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:49.880995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:49.881278 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:49.881548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:49.901114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:49.902261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:49.910638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:49.963112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1125/2255] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:49.970925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:49.971206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:49.971511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:49.990638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:49.991820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:49.999473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1131/2256] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:50.003930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.013426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670b831-1bf2-43e1-bd4c-58ff85604eb7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.017872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.019143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.019413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.019678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.020311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:09:50.020604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.022203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.033590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1126/2257] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:09:50.037874 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.047186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656b561-4c71-4abc-97d9-cde55eecfaed None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.052099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.052696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.052970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.053266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.053897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.055852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.067852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:50.085671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1132/2258] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:09:50 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:50.089887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.099801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc4426-b38e-45f9-8493-a26ddb0eaa55 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.155172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.155827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.155960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.156228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.156843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.159062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.180476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1127/2259] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:09:50 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:50.184642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.195286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdcd1b-72b9-47e1-9fcf-fb90a549aac3 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.199737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.200322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.200597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.200866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.201499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.203106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.212648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1133/2260] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:50.216503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.226483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757b8b4-ca7b-4314-bfb5-0ab6283f3629 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.231176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f9deb-757d-451c-bc21-1eb837f4f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.231764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f9deb-757d-451c-bc21-1eb837f4f819 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.232043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f9deb-757d-451c-bc21-1eb837f4f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.232321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f9deb-757d-451c-bc21-1eb837f4f819 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.232954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f9deb-757d-451c-bc21-1eb837f4f819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.234602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f9deb-757d-451c-bc21-1eb837f4f819 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.245790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1128/2261] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:50.249669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.259890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f7405-a13d-414d-878b-875c3b4155be admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.264693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.265327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.265603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.265877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] PATH_INFO: `/v3/projects/c88010be9047455094d5498d40c10cfd/users/3e346dd4e035416c947b7e0013f90018/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.281262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88010be9047455094d5498d40c10cfd, user_id=3e346dd4e035416c947b7e0013f90018, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.284213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.296504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1134/2262] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:50 2022] PUT /identity/v3/projects/c88010be9047455094d5498d40c10cfd/users/3e346dd4e035416c947b7e0013f90018/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:50.302010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0c82-12a1-4307-a52c-503966e74956 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.302296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0c82-12a1-4307-a52c-503966e74956 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.302567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff0c82-12a1-4307-a52c-503966e74956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.322283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b2ff0c82-12a1-4307-a52c-503966e74956 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:50.323494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b2ff0c82-12a1-4307-a52c-503966e74956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:50.381112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1129/2263] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:09:50.390802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:50.392642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:50.451798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df3975a-8c52-4536-ab25-9575ed806582 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:50.455603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:50.456261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:50.456541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:50.456823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:50.466087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:50.467509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:50.477345 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1135/2264] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:09:51.092691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.092973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.093246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.121405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] MFA Rules not processed for user `3e346dd4e035416c947b7e0013f90018`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:51.122830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:51.131023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:51.190990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1130/2265] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:51 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:51.195451 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:51.205021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b475a-388d-4ddb-ac5d-4a07ff847c5b service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:51.210202 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:51.210811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.211157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.211492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.212229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:51.214225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:51.241761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1136/2266] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:09:51 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:51.245919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:51.257484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf1e11-a945-44a5-97f2-4d149d0030a3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:51.314163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:51.314773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.315052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.315324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.316473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:51.318422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:51.340213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1131/2267] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:09:51 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:51.344280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:51.354561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07ae70-17c6-4df6-8b3a-c0ad64a5e4f9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:51.359180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:51.359775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.360056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.360329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.360965 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:51.362596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:51.373160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1137/2268] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:51.377347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:51.387665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7ae07-d152-4c85-b0cc-38ea9add5b94 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:51.392234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:51.392871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.393151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.393422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] PATH_INFO: `/v3/projects/ad619c1c63b84d3bb2fe9bb8db7b113c/users/212e607f06f94b468c32c9d57605d619/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.408505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad619c1c63b84d3bb2fe9bb8db7b113c, user_id=212e607f06f94b468c32c9d57605d619, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:51.411240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:51.423789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1132/2269] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:51 2022] PUT /identity/v3/projects/ad619c1c63b84d3bb2fe9bb8db7b113c/users/212e607f06f94b468c32c9d57605d619/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:09:51.427702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:51.437404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39155c0-9741-40c2-9bac-9a1ac6ef575b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:51.493798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98ded0-78d3-4eab-9bff-9cfaaa764009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:51.494362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98ded0-78d3-4eab-9bff-9cfaaa764009 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.494634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98ded0-78d3-4eab-9bff-9cfaaa764009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.494925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98ded0-78d3-4eab-9bff-9cfaaa764009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.495555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98ded0-78d3-4eab-9bff-9cfaaa764009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:51.497231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98ded0-78d3-4eab-9bff-9cfaaa764009 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:51.508096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1138/2270] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:51.512251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:51.522984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c2dd2-e710-4677-ba96-6c84b905a922 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:51.527567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:51.528225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:51.528494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:51.528767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] PATH_INFO: `/v3/projects/ad619c1c63b84d3bb2fe9bb8db7b113c/users/212e607f06f94b468c32c9d57605d619/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:51.530652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad619c1c63b84d3bb2fe9bb8db7b113c, user_id=212e607f06f94b468c32c9d57605d619, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:51.533539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:51.544045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1133/2271] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:51 2022] PUT /identity/v3/projects/ad619c1c63b84d3bb2fe9bb8db7b113c/users/212e607f06f94b468c32c9d57605d619/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:52.177423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:52.177734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:52.178054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:52.206154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] MFA Rules not processed for user `212e607f06f94b468c32c9d57605d619`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:52.207921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:52.218042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:52.282038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1139/2272] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:52 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:09:53.543701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:53.605763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1ac08-8172-40ce-9688-b082940d0a81 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:53.610366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:53.610989 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:53.611293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:53.611563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] PATH_INFO: `/v3/users/3e346dd4e035416c947b7e0013f90018` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:53.612698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e346dd4e035416c947b7e0013f90018)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:53.614643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:53.616890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:53.664339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e346dd4e035416c947b7e0013f90018'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:53.671745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e346dd4e035416c947b7e0013f90018'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:53.672370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e346dd4e035416c947b7e0013f90018'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:53.679375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e346dd4e035416c947b7e0013f90018'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:53.680011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e346dd4e035416c947b7e0013f90018'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:53.697641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1134/2273] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:09:53 2022] DELETE /identity/v3/users/3e346dd4e035416c947b7e0013f90018 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:53.872524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:53.889271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85ed5a-4c87-4836-bc2c-e0117d366daf None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:53.944865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:53.945492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:53.945769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:53.946074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] PATH_INFO: `/v3/projects/c88010be9047455094d5498d40c10cfd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:53.947180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88010be9047455094d5498d40c10cfd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:53.949118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:53.994250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88010be9047455094d5498d40c10cfd', 'request_id': 'req-4c07f1f7-8831-4d84-8267-593dbe08a42e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:53.999695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88010be9047455094d5498d40c10cfd', 'request_id': 'req-4c07f1f7-8831-4d84-8267-593dbe08a42e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.007735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88010be9047455094d5498d40c10cfd', 'request_id': 'req-4c07f1f7-8831-4d84-8267-593dbe08a42e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.018879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] The token cache is being invalidate because project c88010be9047455094d5498d40c10cfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:09:54.019162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.019786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.021237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1140/2274] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:09:53 2022] DELETE /identity/v3/projects/c88010be9047455094d5498d40c10cfd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:54.025228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.026659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:54.039653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d44e-e140-42cc-be36-81ab7968f4e7 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.099378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.100444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.100719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.100996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] PATH_INFO: `/v3/users/212e607f06f94b468c32c9d57605d619` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.102174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=212e607f06f94b468c32c9d57605d619)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.104129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.153102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '212e607f06f94b468c32c9d57605d619'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.160626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '212e607f06f94b468c32c9d57605d619'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.161375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '212e607f06f94b468c32c9d57605d619'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.168378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '212e607f06f94b468c32c9d57605d619'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.169010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f7e4298b-9b65-48a5-b7b3-4451cc56ce28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '212e607f06f94b468c32c9d57605d619'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.182919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:54.188082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1135/2275] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:09:54 2022] DELETE /identity/v3/users/212e607f06f94b468c32c9d57605d619 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:54.357417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.359007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:54.374688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07f1f7-8831-4d84-8267-593dbe08a42e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.413435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:54.439723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.440361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.440636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.440926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] PATH_INFO: `/v3/projects/ad619c1c63b84d3bb2fe9bb8db7b113c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.442066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad619c1c63b84d3bb2fe9bb8db7b113c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.444093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.493701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad619c1c63b84d3bb2fe9bb8db7b113c', 'request_id': 'req-383fbca3-0e1d-47f1-820c-deb52ffee8d4'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.499317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad619c1c63b84d3bb2fe9bb8db7b113c', 'request_id': 'req-383fbca3-0e1d-47f1-820c-deb52ffee8d4'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.506684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad619c1c63b84d3bb2fe9bb8db7b113c', 'request_id': 'req-383fbca3-0e1d-47f1-820c-deb52ffee8d4'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.517641 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] The token cache is being invalidate because project ad619c1c63b84d3bb2fe9bb8db7b113c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:09:54.517920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.518557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-383fbca3-0e1d-47f1-820c-deb52ffee8d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:09:54.520067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1141/2276] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:09:54 2022] DELETE /identity/v3/projects/ad619c1c63b84d3bb2fe9bb8db7b113c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:54.524915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.525203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.525479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.545926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:54.547116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:54.607564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1136/2277] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:54.615320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.615611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.615889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.635117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:54.636305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:54.644080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1142/2278] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:09:54.648610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.659062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e9dc3-d84b-46b2-944b-ae722abdec88 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.664250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.664856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.665134 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.665405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.666085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:09:54.666385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.667968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.679310 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1137/2279] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:09:54.683750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.692941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3d155b-89e2-4a8e-8a2e-aabc017d64d5 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.697538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.698168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.698445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.698716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.699347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.701234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.733852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1143/2280] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:09:54 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:54.738557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.753537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d4514-1c65-49de-8417-812d9f47b9ff admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.809343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.809878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.810188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.810468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.811093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.813037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.834616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1138/2281] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:09:54 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:09:54.838713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.849471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653cc53-bf2f-4d42-a63a-adfdd5a31a68 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.853931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.854554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.854828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.855098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.855723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.857294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.867198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1144/2282] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:54.871087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.880985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2efdf6-fb2e-4444-858c-3bb46c7911db admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.885986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5be132-8546-4972-b970-ce7fe31f653e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.886603 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5be132-8546-4972-b970-ce7fe31f653e admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.886879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5be132-8546-4972-b970-ce7fe31f653e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.887147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5be132-8546-4972-b970-ce7fe31f653e admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.887775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5be132-8546-4972-b970-ce7fe31f653e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.889712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5be132-8546-4972-b970-ce7fe31f653e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.901024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1139/2283] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:09:54.905026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:54.915762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f4868-8036-4465-866f-372f325e5934 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:54.920295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:54.920929 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.921204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.921472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] PATH_INFO: `/v3/projects/a15248a0726242cfb3244f3f97724ae8/users/4c10d49c37c6482ca87e64b662285481/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.935395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15248a0726242cfb3244f3f97724ae8, user_id=4c10d49c37c6482ca87e64b662285481, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:54.938046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:54.951043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1145/2284] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:54 2022] PUT /identity/v3/projects/a15248a0726242cfb3244f3f97724ae8/users/4c10d49c37c6482ca87e64b662285481/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:54.962850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:54.963131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:54.963405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:54.990616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] MFA Rules not processed for user `4c10d49c37c6482ca87e64b662285481`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:54.991816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:55.045719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:55.060333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1140/2285] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:55.064830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.127669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c543-3ce8-4a60-aa5b-abcf2f5f4cff admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.132880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.133486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.133766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.134077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.134700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.136621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.146602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:09:55.169601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1146/2286] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:09:55 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:55.170764 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.181145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86589a58-83a1-45df-8da6-56a508d2f64c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.236668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.237281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.237563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.237836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.238512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.240436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.261448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1141/2287] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:09:55 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:09:55.265502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.276160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fe8d1-5e32-4f69-800b-53aad4efb304 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.280701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.281290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.281563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.281828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.282492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.284321 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.294813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1147/2288] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:55.298768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.309022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61ab32-fefa-4052-9994-8997c3e2b40a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.313601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.314230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.314509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.314781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] PATH_INFO: `/v3/projects/5d44d96d5a17479c9e810eb7a2170dee/users/90bc19f567d041f4afbf74e38740e074/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.329891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d44d96d5a17479c9e810eb7a2170dee, user_id=90bc19f567d041f4afbf74e38740e074, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.332587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.345251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1142/2289] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:55 2022] PUT /identity/v3/projects/5d44d96d5a17479c9e810eb7a2170dee/users/90bc19f567d041f4afbf74e38740e074/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:55.349143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.358984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8014ef-30fd-4093-a5a6-d54b73ddcf5d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.415597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b3a160-6c5f-40b8-9a79-aa67757a2097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.416204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3a160-6c5f-40b8-9a79-aa67757a2097 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.416485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3a160-6c5f-40b8-9a79-aa67757a2097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.416761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b3a160-6c5f-40b8-9a79-aa67757a2097 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.417539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3a160-6c5f-40b8-9a79-aa67757a2097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.419162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b3a160-6c5f-40b8-9a79-aa67757a2097 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.429227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1148/2290] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:09:55.433311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.444008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68deef66-261e-449d-984a-06354ff51d1a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.449522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.450209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.450491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.450762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] PATH_INFO: `/v3/projects/5d44d96d5a17479c9e810eb7a2170dee/users/90bc19f567d041f4afbf74e38740e074/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.452663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d44d96d5a17479c9e810eb7a2170dee, user_id=90bc19f567d041f4afbf74e38740e074, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.455360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.465484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1143/2291] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:09:55 2022] PUT /identity/v3/projects/5d44d96d5a17479c9e810eb7a2170dee/users/90bc19f567d041f4afbf74e38740e074/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:09:55.475420 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.475713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.475989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.508916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] MFA Rules not processed for user `90bc19f567d041f4afbf74e38740e074`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:09:55.510150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:55.577609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1149/2292] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:09:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:09:55.588293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:55.590022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:55.645550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5aa72-0239-4ac2-a94e-bdbd10ab6c2d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:55.649193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:55.649794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:55.650103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:55.650375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:55.705795 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:55.707266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:55.716282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1144/2293] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:09:59.924088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:09:59.925497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:09:59.937490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64e89c-a5cb-408c-9ebe-996664ca3a36 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:09:59.941104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8920385-9f58-4bff-9589-a2632dcfe1c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:09:59.941711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920385-9f58-4bff-9589-a2632dcfe1c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:09:59.941988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920385-9f58-4bff-9589-a2632dcfe1c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:09:59.942278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8920385-9f58-4bff-9589-a2632dcfe1c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:09:59.952223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920385-9f58-4bff-9589-a2632dcfe1c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:09:59.953648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8920385-9f58-4bff-9589-a2632dcfe1c7 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:09:59.963182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1150/2294] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:09:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:04.924968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:04.929747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:04.992393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2e63e-28c5-46b4-b75f-abc85a72ca38 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:04.996921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:04.997563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:04.997842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:04.998151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] PATH_INFO: `/v3/users/4c10d49c37c6482ca87e64b662285481` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:04.999278 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c10d49c37c6482ca87e64b662285481)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:05.001298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:05.052582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c10d49c37c6482ca87e64b662285481'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.061056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c10d49c37c6482ca87e64b662285481'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.061700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c10d49c37c6482ca87e64b662285481'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.069502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c10d49c37c6482ca87e64b662285481'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.070164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c10d49c37c6482ca87e64b662285481'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.088904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1145/2295] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:10:04 2022] DELETE /identity/v3/users/4c10d49c37c6482ca87e64b662285481 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:10:05.093480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:05.093962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:05.094094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:05.113372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:05.114629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:05.162616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:05.184078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1151/2296] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:05.196169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:05.197826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:05.259734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f79da83-0453-4512-844c-294750f56527 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:05.263442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:05.264052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:05.264336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:05.264613 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:05.274585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:05.276026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:05.287961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1146/2297] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:05.723072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:05.724508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:05.747805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8b4627-bd07-489d-a0d0-1e41af27b744 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:05.753561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:05.754225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:05.754515 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:05.754890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] PATH_INFO: `/v3/projects/a15248a0726242cfb3244f3f97724ae8` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:05.756117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15248a0726242cfb3244f3f97724ae8)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:05.758044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:05.809616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15248a0726242cfb3244f3f97724ae8', 'request_id': 'req-d74d360e-d6b5-47e6-952f-b335e332421c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.815811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15248a0726242cfb3244f3f97724ae8', 'request_id': 'req-d74d360e-d6b5-47e6-952f-b335e332421c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.823936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15248a0726242cfb3244f3f97724ae8', 'request_id': 'req-d74d360e-d6b5-47e6-952f-b335e332421c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.836713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] The token cache is being invalidate because project a15248a0726242cfb3244f3f97724ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:10:05.837124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.837818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d74d360e-d6b5-47e6-952f-b335e332421c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.839361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1152/2298] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:10:05 2022] DELETE /identity/v3/projects/a15248a0726242cfb3244f3f97724ae8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:05.843373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:05.853605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203552e-5743-4c48-b472-c0a622354362 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:05.874050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:05.915473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:05.916109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:05.916390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:05.917057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] PATH_INFO: `/v3/users/90bc19f567d041f4afbf74e38740e074` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:05.917826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bc19f567d041f4afbf74e38740e074)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:05.919787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:05.966442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bc19f567d041f4afbf74e38740e074'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.974315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bc19f567d041f4afbf74e38740e074'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.974980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bc19f567d041f4afbf74e38740e074'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.982710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bc19f567d041f4afbf74e38740e074'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:05.983344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bc19f567d041f4afbf74e38740e074'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:06.000068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1147/2299] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:10:05 2022] DELETE /identity/v3/users/90bc19f567d041f4afbf74e38740e074 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:10:06.072217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6181-15c1-4a0f-a89b-ec5421b54636 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.072528 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6181-15c1-4a0f-a89b-ec5421b54636 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.072847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6181-15c1-4a0f-a89b-ec5421b54636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.093046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-914b6181-15c1-4a0f-a89b-ec5421b54636 None None] MFA Rules not processed for user `4b1b0448e7f54e3aaae49e411699269e`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:06.094442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-914b6181-15c1-4a0f-a89b-ec5421b54636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:06.163761 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1153/2300] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:10:06.332518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.334019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:06.402648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e771d-33dd-47ab-ac1a-0b6a9d057f0c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.407188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.407829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.408104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.408629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] PATH_INFO: `/v3/projects/5d44d96d5a17479c9e810eb7a2170dee` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.409782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d44d96d5a17479c9e810eb7a2170dee)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.411738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.462221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:06.465558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d44d96d5a17479c9e810eb7a2170dee', 'request_id': 'req-575833ab-23c9-440b-9800-bedb8ada0360'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:06.475394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d44d96d5a17479c9e810eb7a2170dee', 'request_id': 'req-575833ab-23c9-440b-9800-bedb8ada0360'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:06.480949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d44d96d5a17479c9e810eb7a2170dee', 'request_id': 'req-575833ab-23c9-440b-9800-bedb8ada0360'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:06.492178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] The token cache is being invalidate because project 5d44d96d5a17479c9e810eb7a2170dee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:10:06.492467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:06.493114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-575833ab-23c9-440b-9800-bedb8ada0360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:06.494681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1148/2301] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:10:06 2022] DELETE /identity/v3/projects/5d44d96d5a17479c9e810eb7a2170dee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:06.500701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.500989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.501266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.536166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:06.537393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:06.605492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1154/2302] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:06.608162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.608506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.608789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.628775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:06.629990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:06.638604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1149/2303] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:06.642945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.653828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759ff1b-bed5-4fd8-b654-dc92ff185965 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.658998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.659923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.660206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.660482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.661138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:10:06.661449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.663217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.675139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1155/2304] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:10:06.680067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.689229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4db602-2e7a-4232-ba8e-1fa94d740826 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.694583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.695169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.695445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.695716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.696356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.698289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.729748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1150/2305] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:10:06 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:06.730801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.735950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:06.742992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac07b29-6cb9-477a-aca5-6fa49580e1d4 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.801581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.802222 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.802502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.802775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.803952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.805861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.829737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1156/2306] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:10:06 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:06.834102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.845689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0fdc8-99c3-4699-8667-b01741d6b558 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.851250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.852027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.852402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.852773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.853518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.855252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.867731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1151/2307] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:10:06.872007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.883802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6474a-f40b-4620-9d8b-1a0b989ebe45 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.888708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52033773-9d38-4c5b-adbf-ea1499987c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.889408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52033773-9d38-4c5b-adbf-ea1499987c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.889712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52033773-9d38-4c5b-adbf-ea1499987c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.890080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52033773-9d38-4c5b-adbf-ea1499987c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.890808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52033773-9d38-4c5b-adbf-ea1499987c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.892497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52033773-9d38-4c5b-adbf-ea1499987c13 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.903833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1157/2308] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:06.907749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:06.918342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc8ad71-221b-4a0e-9c2e-c36114fa4f55 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:06.923127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:06.923784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.924063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.924333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] PATH_INFO: `/v3/projects/17d4f502b14b456082c34f9dfe536694/users/343d8dfe6bc041878ad149e56b319562/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.939208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d4f502b14b456082c34f9dfe536694, user_id=343d8dfe6bc041878ad149e56b319562, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:06.941965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:06.957393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1152/2309] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:06 2022] PUT /identity/v3/projects/17d4f502b14b456082c34f9dfe536694/users/343d8dfe6bc041878ad149e56b319562/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:06.968821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:06.969213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:06.969449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:06.996619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] MFA Rules not processed for user `343d8dfe6bc041878ad149e56b319562`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:06.997723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:07.065693 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1158/2310] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:07.077638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:07.079401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:07.138268 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d35ce9-4190-480c-8f19-24e2206c3336 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:07.142150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e23a07-37a3-4187-ad67-29965bdd5453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:07.142762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e23a07-37a3-4187-ad67-29965bdd5453 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:07.143038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e23a07-37a3-4187-ad67-29965bdd5453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:07.143305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e23a07-37a3-4187-ad67-29965bdd5453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:07.152762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e23a07-37a3-4187-ad67-29965bdd5453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:07.154228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e23a07-37a3-4187-ad67-29965bdd5453 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:07.163876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1153/2311] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:16.596735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:16.661389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02416c9f-29c9-462c-b0c4-2f10b99194af None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:16.665981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:16.666625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:16.666903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:16.667200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] PATH_INFO: `/v3/users/343d8dfe6bc041878ad149e56b319562` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:16.668314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=343d8dfe6bc041878ad149e56b319562)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:16.670266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:16.682308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:16.714942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343d8dfe6bc041878ad149e56b319562'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:16.722765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343d8dfe6bc041878ad149e56b319562'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:16.723392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343d8dfe6bc041878ad149e56b319562'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:16.724024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343d8dfe6bc041878ad149e56b319562'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:16.730842 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343d8dfe6bc041878ad149e56b319562'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:16.747233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1159/2312] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:10:16 2022] DELETE /identity/v3/users/343d8dfe6bc041878ad149e56b319562 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:16.751677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:16.752132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:16.752327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:16.771066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:16.772248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:16.830941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1154/2313] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:10:16.841059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:16.842467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:16.897196 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fbce4-a47f-410c-abcb-b74a45b1a480 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:16.901523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a5088-b25f-4ef1-a5e8-34cdf31daaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:16.902168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a5088-b25f-4ef1-a5e8-34cdf31daaaa service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:16.902448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a5088-b25f-4ef1-a5e8-34cdf31daaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:16.902719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a5088-b25f-4ef1-a5e8-34cdf31daaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:16.911844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a5088-b25f-4ef1-a5e8-34cdf31daaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:16.913263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990a5088-b25f-4ef1-a5e8-34cdf31daaaa service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:16.922743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1160/2314] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:17.184115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.185756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:17.198473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb3cb5-a5b1-4169-ab18-49710cddbac4 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.203221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.203916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.204135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.204407 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] PATH_INFO: `/v3/projects/17d4f502b14b456082c34f9dfe536694` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.205532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d4f502b14b456082c34f9dfe536694)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.207549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.216564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:17.257146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d4f502b14b456082c34f9dfe536694', 'request_id': 'req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:17.265446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d4f502b14b456082c34f9dfe536694', 'request_id': 'req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:17.271576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d4f502b14b456082c34f9dfe536694', 'request_id': 'req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:17.282618 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] The token cache is being invalidate because project 17d4f502b14b456082c34f9dfe536694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:10:17.282902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:17.283541 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-32d89ad9-e9ae-4fed-b9cc-2c1e10786e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:17.285544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1155/2315] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:10:17 2022] DELETE /identity/v3/projects/17d4f502b14b456082c34f9dfe536694 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:17.290093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.290482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.290833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.310423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:17.311625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:17.378117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1161/2316] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:17.386220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.386543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.386827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.408769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:17.408769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:17.416886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1156/2317] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:17.421182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.430574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59685d40-c492-4a81-87f5-c702b4ea1041 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.435951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.436552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.436829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.437098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.437737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:10:17.438061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.439648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.451517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1162/2318] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:10:17.456088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.465522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05914139-1a70-4988-b26e-fbbf62de2019 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.470218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.470831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.471138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.471438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.472136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.474069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.503908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1157/2319] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:10:17 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:17.508158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.518341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a559dc-5acf-48ec-8433-de69e85c851a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.536056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:17.582702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.583368 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.583650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.583931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.584581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.586622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.614111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1163/2320] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:10:17 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:17.615669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.625832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f00e-7668-439c-84ab-c5cc720869fb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.630526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.631067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.631347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.631620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.632260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.633850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.644364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1158/2321] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:17.648323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.659347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dbff25-7990-461c-a0fd-aedb7a4b4d78 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.663957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949215d-b0f2-477e-82fe-d750205c4e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.664556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949215d-b0f2-477e-82fe-d750205c4e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.664850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949215d-b0f2-477e-82fe-d750205c4e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.665113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949215d-b0f2-477e-82fe-d750205c4e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.665750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949215d-b0f2-477e-82fe-d750205c4e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.667382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949215d-b0f2-477e-82fe-d750205c4e39 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.677795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1164/2322] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:17.682023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.691763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae15035-e183-42bd-8efa-dde825961215 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.696323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.697526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.697803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.698122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] PATH_INFO: `/v3/projects/f5f64a562caa4c8b9a891cf4e1630dfb/users/17991f5c788441d2a8b8bcad3e2f53cd/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.712026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f64a562caa4c8b9a891cf4e1630dfb, user_id=17991f5c788441d2a8b8bcad3e2f53cd, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.715250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.727687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1159/2323] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:17 2022] PUT /identity/v3/projects/f5f64a562caa4c8b9a891cf4e1630dfb/users/17991f5c788441d2a8b8bcad3e2f53cd/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:17.738785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.739075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.739352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.767186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] MFA Rules not processed for user `17991f5c788441d2a8b8bcad3e2f53cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:17.768391 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:17.833488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1165/2324] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:17.866950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:17.869300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:17.925832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a4d58-38bf-4ef0-8ad8-f2df923d37f8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:17.929276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d7bac-f7f8-442c-9c6d-a24227a38c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:17.929874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d7bac-f7f8-442c-9c6d-a24227a38c24 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:17.930191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d7bac-f7f8-442c-9c6d-a24227a38c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:17.930465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d7bac-f7f8-442c-9c6d-a24227a38c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:17.940359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d7bac-f7f8-442c-9c6d-a24227a38c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:17.941840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d7bac-f7f8-442c-9c6d-a24227a38c24 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:17.951460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1160/2325] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:17.966916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.032063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf8be-a801-483c-bd13-33fffc7a274c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.036937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.037565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.037840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.038741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] PATH_INFO: `/v3/users/17991f5c788441d2a8b8bcad3e2f53cd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.039886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17991f5c788441d2a8b8bcad3e2f53cd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:18.041821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:18.075541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:18.090703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17991f5c788441d2a8b8bcad3e2f53cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.097525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17991f5c788441d2a8b8bcad3e2f53cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.098188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17991f5c788441d2a8b8bcad3e2f53cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.098792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17991f5c788441d2a8b8bcad3e2f53cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.106116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17991f5c788441d2a8b8bcad3e2f53cd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.123404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1166/2326] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:10:17 2022] DELETE /identity/v3/users/17991f5c788441d2a8b8bcad3e2f53cd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:18.127861 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.128142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.128417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.146875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:18.148067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:18.177439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:18.208654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1161/2327] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:18.218265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.219711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:18.276324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e232a1-4a80-4a0c-9009-9d512bac0f02 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.279974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de58d35-7ea9-4f86-87c0-3f2fd2baa1a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.280572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58d35-7ea9-4f86-87c0-3f2fd2baa1a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.280852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58d35-7ea9-4f86-87c0-3f2fd2baa1a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.281125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de58d35-7ea9-4f86-87c0-3f2fd2baa1a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.290730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58d35-7ea9-4f86-87c0-3f2fd2baa1a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:18.292147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de58d35-7ea9-4f86-87c0-3f2fd2baa1a3 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:18.301482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1167/2328] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:18.580190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.581648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:18.594321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f0e2a-1ef4-46cd-812b-61372cbf8d5f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.598969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.599593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.599872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.600138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] PATH_INFO: `/v3/projects/f5f64a562caa4c8b9a891cf4e1630dfb` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.601253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f64a562caa4c8b9a891cf4e1630dfb)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:18.603739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:18.652033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f64a562caa4c8b9a891cf4e1630dfb', 'request_id': 'req-93b42d6e-9618-4eb6-a976-19f013c0c08e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.660146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f64a562caa4c8b9a891cf4e1630dfb', 'request_id': 'req-93b42d6e-9618-4eb6-a976-19f013c0c08e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.666271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f64a562caa4c8b9a891cf4e1630dfb', 'request_id': 'req-93b42d6e-9618-4eb6-a976-19f013c0c08e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.676970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] The token cache is being invalidate because project f5f64a562caa4c8b9a891cf4e1630dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:10:18.677254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.677871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-93b42d6e-9618-4eb6-a976-19f013c0c08e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:18.679517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1162/2329] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:10:18 2022] DELETE /identity/v3/projects/f5f64a562caa4c8b9a891cf4e1630dfb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:18.684230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.684706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.684800 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.704190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:18.705372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:18.761646 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1168/2330] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:18.769331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.769614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.769888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.789926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:18.791776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:18.799559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1163/2331] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:18.803425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.812830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73c89f-fc29-416a-bff2-2b27bff46b3c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.817389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.817970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.818273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.818544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.819178 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:10:18.819496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:18.821073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:18.832930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1169/2332] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:10:18.836995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.846578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1aa2f7-57e7-4b94-b4bb-667deb9b0ccf None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.851190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.851781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.852057 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.852328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.852997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:18.854911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:18.882611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1164/2333] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:10:18 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:10:18.887237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.897137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809de74c-6ae3-415a-8d80-8561ef29da19 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.930101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:18.954757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.955355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.955629 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.955898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:18.956512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:18.958469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:18.981696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1170/2334] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:10:18 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:18.983691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:18.994270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965776a-c6cb-4c64-9014-b0fe2ee01edb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:18.998813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:18.999403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:18.999679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:18.999950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.000584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.002195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.012573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1165/2335] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:19.016383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.026597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeeaecf-004e-45f4-95ed-434f35cfad40 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.031168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5750d-d821-4110-8c92-2ac8692a862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.031748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5750d-d821-4110-8c92-2ac8692a862b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.032025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5750d-d821-4110-8c92-2ac8692a862b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.032289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5750d-d821-4110-8c92-2ac8692a862b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.032908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5750d-d821-4110-8c92-2ac8692a862b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.034786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5750d-d821-4110-8c92-2ac8692a862b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.044314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1171/2336] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:19.048126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.057967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65efee48-c071-4374-9e4f-a6612c5341cf admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.062483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.063114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.063390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.063659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] PATH_INFO: `/v3/projects/88f10162d3624b099ef7578fa9de202a/users/756dd835fc374bc7ad0e0a9c68aa4d65/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.078112 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f10162d3624b099ef7578fa9de202a, user_id=756dd835fc374bc7ad0e0a9c68aa4d65, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.080770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.094233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1166/2337] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:19 2022] PUT /identity/v3/projects/88f10162d3624b099ef7578fa9de202a/users/756dd835fc374bc7ad0e0a9c68aa4d65/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:19.104358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.104639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.104910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.131660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] MFA Rules not processed for user `756dd835fc374bc7ad0e0a9c68aa4d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:19.132848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:19.195382 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1172/2338] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:19.200355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.258576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:19.263798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f581461-000d-42ab-b3b9-35a1fd777b8e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.269902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.270599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.271070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.271392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.272249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.274381 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.300676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1167/2339] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:10:19 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:19.304701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.315920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0ce49-edd8-4e16-8201-a8e544d59c73 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.372208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.372825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.373106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.373381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.374035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.375975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.398352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1173/2340] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:10:19 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:19.402633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.413811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30724ec-74c7-4c48-a227-2f41e63f1ab1 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.418453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.419060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.419329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.419606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.420227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.421811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.432899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1168/2341] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:19.437170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.448826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b465d44-c743-4ac9-9fd4-273e8df6fa8f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.454509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.455171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.455447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.455718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] PATH_INFO: `/v3/projects/ec231de841de4da5be7c854819cc74fc/users/00582aae9bfb474c80858dc5e48b7c55/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.470962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec231de841de4da5be7c854819cc74fc, user_id=00582aae9bfb474c80858dc5e48b7c55, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.473696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.488102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1174/2342] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:19 2022] PUT /identity/v3/projects/ec231de841de4da5be7c854819cc74fc/users/00582aae9bfb474c80858dc5e48b7c55/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:10:19.492053 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.502642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b729199-fe17-4fa1-aa11-2b0d75d61640 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.563051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affe9d4-9ac0-4178-8402-6b3ba9b3b403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.563686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affe9d4-9ac0-4178-8402-6b3ba9b3b403 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.563972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affe9d4-9ac0-4178-8402-6b3ba9b3b403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.564250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affe9d4-9ac0-4178-8402-6b3ba9b3b403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.564911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affe9d4-9ac0-4178-8402-6b3ba9b3b403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.566565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affe9d4-9ac0-4178-8402-6b3ba9b3b403 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.578517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1169/2343] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:19.582073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.593697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a3d58-52c5-4085-96ef-67bef14f8eb2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.598904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.599574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.599860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.600530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] PATH_INFO: `/v3/projects/ec231de841de4da5be7c854819cc74fc/users/00582aae9bfb474c80858dc5e48b7c55/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.602322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec231de841de4da5be7c854819cc74fc, user_id=00582aae9bfb474c80858dc5e48b7c55, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.605495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.611615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:19.619415 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1175/2344] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:19 2022] PUT /identity/v3/projects/ec231de841de4da5be7c854819cc74fc/users/00582aae9bfb474c80858dc5e48b7c55/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:19.688817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.688817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.689121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.720315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] MFA Rules not processed for user `00582aae9bfb474c80858dc5e48b7c55`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:19.721639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:19.801455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1170/2345] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:19.812470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:19.814316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:19.873371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8720be1-c821-4759-a0fc-8cda3aed8667 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:19.877049 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278d014-1bc0-4eef-828e-aafaea9ebf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:19.877653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278d014-1bc0-4eef-828e-aafaea9ebf52 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:19.877935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278d014-1bc0-4eef-828e-aafaea9ebf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:19.878240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278d014-1bc0-4eef-828e-aafaea9ebf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:19.937148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278d014-1bc0-4eef-828e-aafaea9ebf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:19.938620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278d014-1bc0-4eef-828e-aafaea9ebf52 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:19.947397 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1176/2346] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:22.492873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:22.545530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:22.558752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef7197-a8c3-4630-8f39-637389a23f62 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:22.563276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:22.563941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:22.564219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:22.564490 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] PATH_INFO: `/v3/users/756dd835fc374bc7ad0e0a9c68aa4d65` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:22.565606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756dd835fc374bc7ad0e0a9c68aa4d65)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:22.567585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:22.617583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756dd835fc374bc7ad0e0a9c68aa4d65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:22.626209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756dd835fc374bc7ad0e0a9c68aa4d65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:22.626943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756dd835fc374bc7ad0e0a9c68aa4d65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:22.634092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756dd835fc374bc7ad0e0a9c68aa4d65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:22.634736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756dd835fc374bc7ad0e0a9c68aa4d65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:22.653517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1171/2347] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:10:22 2022] DELETE /identity/v3/users/756dd835fc374bc7ad0e0a9c68aa4d65 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:22.657521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:22.657936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:22.658117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:22.677723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:22.678963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:22.738961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1177/2348] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:22.749070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:22.750532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:22.810594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f152-af76-4da3-940e-3b48a1de5387 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:22.814314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:22.814915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:22.815191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:22.815461 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:22.824253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:22.825716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:22.835537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1172/2349] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:23.106637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:23.108162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:23.122960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ef23d2-05ff-4645-be49-d901d3e09cfa None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:23.127939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:23.128579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:23.128861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:23.129137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] PATH_INFO: `/v3/projects/88f10162d3624b099ef7578fa9de202a` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:23.130831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f10162d3624b099ef7578fa9de202a)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:23.132746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:23.188880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:23.192454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f10162d3624b099ef7578fa9de202a', 'request_id': 'req-4f2e71cf-a341-4dd6-9fac-fb1780872c41'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.200763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f10162d3624b099ef7578fa9de202a', 'request_id': 'req-4f2e71cf-a341-4dd6-9fac-fb1780872c41'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.211734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f10162d3624b099ef7578fa9de202a', 'request_id': 'req-4f2e71cf-a341-4dd6-9fac-fb1780872c41'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.224283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] The token cache is being invalidate because project 88f10162d3624b099ef7578fa9de202a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:10:23.224636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.225362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.227156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1178/2350] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:10:23 2022] DELETE /identity/v3/projects/88f10162d3624b099ef7578fa9de202a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:23.231512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:23.242982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91cfca-35b1-4521-9902-90064e19592f service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:23.307438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:23.308074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:23.308350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:23.308620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] PATH_INFO: `/v3/users/00582aae9bfb474c80858dc5e48b7c55` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:23.309741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00582aae9bfb474c80858dc5e48b7c55)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:23.311692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:23.327252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:23.367601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00582aae9bfb474c80858dc5e48b7c55'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.375444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00582aae9bfb474c80858dc5e48b7c55'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.376226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00582aae9bfb474c80858dc5e48b7c55'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.387299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00582aae9bfb474c80858dc5e48b7c55'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.388043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-90dc0d51-44ed-451b-b30f-a28fe581576c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00582aae9bfb474c80858dc5e48b7c55'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.412964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1173/2351] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:10:23 2022] DELETE /identity/v3/users/00582aae9bfb474c80858dc5e48b7c55 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:23.855983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:23.857566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:23.872162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e71cf-a341-4dd6-9fac-fb1780872c41 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:23.931437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:23.932600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:23.932876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:23.933144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] PATH_INFO: `/v3/projects/ec231de841de4da5be7c854819cc74fc` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:23.934332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec231de841de4da5be7c854819cc74fc)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:23.936658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:23.984346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec231de841de4da5be7c854819cc74fc', 'request_id': 'req-55f1a9de-f200-4138-944e-55ef82a287ee'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.989659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec231de841de4da5be7c854819cc74fc', 'request_id': 'req-55f1a9de-f200-4138-944e-55ef82a287ee'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:23.997939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec231de841de4da5be7c854819cc74fc', 'request_id': 'req-55f1a9de-f200-4138-944e-55ef82a287ee'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:24.010078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] The token cache is being invalidate because project ec231de841de4da5be7c854819cc74fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:10:24.010367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:24.011000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-55f1a9de-f200-4138-944e-55ef82a287ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:10:24.012566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1179/2352] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:10:23 2022] DELETE /identity/v3/projects/ec231de841de4da5be7c854819cc74fc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:10:24.019043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.019509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.019725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.039494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:24.040818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:24.102107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1174/2353] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:24.110257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.110542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.110813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.131937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:24.133125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:24.141459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1180/2354] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:10:24.145663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.155972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa1c76-b1d7-417a-b113-082b16233ae9 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.160917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.161552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.161831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.162158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.162804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:10:24.163108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.164751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.178415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1175/2355] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:10:24.182895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.192862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056c945-98ca-4c06-a637-6cafd9127ebe None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.200859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.201481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.201897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.203206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.203206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.205093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.234446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1181/2356] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:10:24 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:24.238602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.249573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e3b3b-aec4-4989-aea9-8091be045d3f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.310110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.310734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.311010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.311281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.311925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.313854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.340552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1176/2357] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:10:24 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:24.341808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.352080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a47a2-c4a6-4a54-85cc-6855da05431e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.356884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.357488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.357767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.358090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.358718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.360319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.370725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1182/2358] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:24.374773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.386980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d76d-592c-40b2-a43f-24c7ab8fe6b9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.391720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242e2a9-63ab-4e10-ae88-5d3c4f8a2de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.392484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242e2a9-63ab-4e10-ae88-5d3c4f8a2de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.392798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242e2a9-63ab-4e10-ae88-5d3c4f8a2de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.393173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242e2a9-63ab-4e10-ae88-5d3c4f8a2de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.393848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242e2a9-63ab-4e10-ae88-5d3c4f8a2de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.395513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242e2a9-63ab-4e10-ae88-5d3c4f8a2de2 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.398166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:24.410082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1177/2359] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:10:24.414447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.424885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5942ce-d5bc-4b4f-b865-da4170b20d14 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.429855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.430537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.430821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.431168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] PATH_INFO: `/v3/projects/85c73f9a74da4079a54a503e0b42ca2e/users/f67fe549bae6447f8b24219becb3bf97/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.445427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c73f9a74da4079a54a503e0b42ca2e, user_id=f67fe549bae6447f8b24219becb3bf97, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.448731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.463840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1183/2360] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:24 2022] PUT /identity/v3/projects/85c73f9a74da4079a54a503e0b42ca2e/users/f67fe549bae6447f8b24219becb3bf97/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:24.469175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eea1b-cb4e-4df4-bac6-62caa9fcc06d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.469462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eea1b-cb4e-4df4-bac6-62caa9fcc06d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.469759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384eea1b-cb4e-4df4-bac6-62caa9fcc06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.488279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-384eea1b-cb4e-4df4-bac6-62caa9fcc06d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:24.489486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-384eea1b-cb4e-4df4-bac6-62caa9fcc06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:24.550301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1178/2361] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:24.562009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:24.564079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:24.570642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:24.625031 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0966-3c42-4128-a270-d787f89ed5c0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:24.628662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:24.629261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:24.629540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:24.629807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:24.639390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:24.640809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:24.650533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1184/2362] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:28.519351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:28.519885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:28.520082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:28.556813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] MFA Rules not processed for user `f67fe549bae6447f8b24219becb3bf97`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:28.558280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:28.626674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1179/2363] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:28.631175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:28.641809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15495c2d-86ec-4a7f-8a23-283f50fbf9aa service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:28.657751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:28.659718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:28.660005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:28.660263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:28.660930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:28.668099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:28.700764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1185/2364] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:10:28 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:28.705209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:28.717373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9529b-eeb2-48f4-a742-2d44142f9521 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:28.801142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:28.801773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:28.802078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:28.802354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:28.803005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:28.804979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:28.836201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1180/2365] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:10:28 2022] POST /identity/v3/users => generated 299 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:10:28.838205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:28.852943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ba63a-5d4d-4623-8d65-0efa4318ccc8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:28.858757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:28.859378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:28.859658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:28.859932 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:28.860576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:28.862282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:28.874792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1186/2366] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:28.879025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:28.889987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aecf87-be0a-4d36-b401-9c63b60070fe admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:28.894921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:28.895750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:28.896040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:28.896315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] PATH_INFO: `/v3/projects/ca0079034881467cb46e90bd21519543/users/b42b2508925a4f9d9e17ce310a03d6e0/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:28.914422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0079034881467cb46e90bd21519543, user_id=b42b2508925a4f9d9e17ce310a03d6e0, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:28.917294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:28.919647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:28.932388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1181/2367] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:28 2022] PUT /identity/v3/projects/ca0079034881467cb46e90bd21519543/users/b42b2508925a4f9d9e17ce310a03d6e0/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:10:28.936397 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:28.947144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f0e66-bee4-43b1-ac82-f4565c36fc4d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:29.005305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6aa39-169b-4e8c-8e12-cfdb95021516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:29.005841 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6aa39-169b-4e8c-8e12-cfdb95021516 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:29.006149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6aa39-169b-4e8c-8e12-cfdb95021516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:29.006422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6aa39-169b-4e8c-8e12-cfdb95021516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:29.007062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6aa39-169b-4e8c-8e12-cfdb95021516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:29.008660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6aa39-169b-4e8c-8e12-cfdb95021516 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:29.019574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1187/2368] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:10:29.023849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:29.034852 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfdb31-cb25-4e5a-9e72-25c0c75a1167 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:29.039656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:29.040367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:29.040652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:29.040927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] PATH_INFO: `/v3/projects/ca0079034881467cb46e90bd21519543/users/b42b2508925a4f9d9e17ce310a03d6e0/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:29.042893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca0079034881467cb46e90bd21519543, user_id=b42b2508925a4f9d9e17ce310a03d6e0, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:29.045550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:29.057361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1182/2369] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:10:29 2022] PUT /identity/v3/projects/ca0079034881467cb46e90bd21519543/users/b42b2508925a4f9d9e17ce310a03d6e0/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:10:33.558165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8456-4b17-4e90-b4e4-d76dbaf188a7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:33.558711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8456-4b17-4e90-b4e4-d76dbaf188a7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:33.558711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d8456-4b17-4e90-b4e4-d76dbaf188a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:33.587803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-086d8456-4b17-4e90-b4e4-d76dbaf188a7 None None] MFA Rules not processed for user `b42b2508925a4f9d9e17ce310a03d6e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:33.589036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-086d8456-4b17-4e90-b4e4-d76dbaf188a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:33.648352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:33.668510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1188/2370] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:33.681377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:33.683024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:33.743578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d74ca9-a724-4e83-b858-c14e56587504 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:33.747369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:33.747996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:33.748282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:33.748558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:33.809118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:33.810686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:33.820089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1183/2371] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:33.969456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc9ee9-caa4-46c0-940e-5604e6cf0e1d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:33.969835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc9ee9-caa4-46c0-940e-5604e6cf0e1d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:33.970203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc9ee9-caa4-46c0-940e-5604e6cf0e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:33.991896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-87dc9ee9-caa4-46c0-940e-5604e6cf0e1d None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:33.993258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-87dc9ee9-caa4-46c0-940e-5604e6cf0e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:34.049898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1189/2372] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:10:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:34.095719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:34.097125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:34.165677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678d53a-9ab2-4b4c-9ce4-eedd3f9a95d1 service glance] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:34.170554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:34.171157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:34.171429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:34.171697 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:34.179436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:34.181002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:34.189039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1184/2373] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:10:43.100782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:43.101285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:43.101400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:43.120147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] MFA Rules not processed for user `f67fe549bae6447f8b24219becb3bf97`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:43.121408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:43.128209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1190/2374] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:10:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:43.169546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:43.178798 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32942f9-3dea-40ab-a1c0-9a883779c955 service swift] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:43.182844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:43.183429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:43.183700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:43.183965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:43.185677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:43.208615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:43.257537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:43.259003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509d33c-0bf8-4f41-a620-a12041486064 service glance] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:43.270540 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1185/2375] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:10:43.812274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:43.868124 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af328ab-5e03-4e4e-96ff-d88adf0dab1f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:43.872324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:43.872947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:43.873223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:43.873494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:43.875288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:43.888706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:43.890158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:43.900974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1191/2376] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:10:45.261939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:45.262441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:45.262656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:45.281863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:10:45.283091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:45.290377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1186/2377] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:10:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:10:45.303894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:45.315958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595aeed-62aa-4958-b533-df80f34eaf99 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:45.321371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:45.322117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:45.322399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:45.322674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:45.331054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:45.332692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:45.344003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1192/2378] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:10:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:10:55.276581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=26 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:55.337634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=63 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:10:59.280488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:10:59.281874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:59.298132 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33482-bc91-4ca6-b9f0-01e768f52785 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:10:59.303607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:10:59.304213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:10:59.304494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:10:59.304767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:10:59.306495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:10:59.384009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:10:59.386086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:10:59.404264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1187/2379] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:10:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:01.575090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:01.576681 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:01.588653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf0a69-55d5-40a4-b96b-adeb33f8aace service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:01.592386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:01.592981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:01.593253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:01.593520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:01.595230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:01.676327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:01.677799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:01.687843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1193/2380] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:11:06.263448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:06.272595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc6df04-1f13-45e6-a3bc-6d53e7ea65db service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:06.276167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:06.276767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:06.277041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:06.277314 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:06.279056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:06.288511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:06.289920 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a2166-9d63-4a1d-bf4a-61e20883f868 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:06.298288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1188/2381] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:11.623738 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:11.634210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872579f-41a2-4dc7-840f-26e9a53209af service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:11.637938 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:11.638662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:11.639007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:11.639347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:11.641186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:11.711327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:11.713308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:11.723306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1194/2382] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:12.247024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:12.247549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:12.247779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:12.269427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] MFA Rules not processed for user `60edcebcccaa4da99ffddadd3e27c1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:11:12.271220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:12.335601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1189/2383] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:11:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:11:13.225080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:13.226608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:13.315957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18bc54-e42e-457e-80ba-3d0ecf680c72 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:13.321280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:13.322278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:13.322787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:13.323367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:13.325805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:13.338280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:13.340010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:13.349985 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1195/2384] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:19.087423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:19.087423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:19.111347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917a1fc-9862-4ea9-9b48-2b5668a2b4a0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:19.116280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:19.117255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:19.117746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:19.118921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] PATH_INFO: `/v3/users/c8fe3c5c28964b768b2c0023fc62f163` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:19.120617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8fe3c5c28964b768b2c0023fc62f163)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:19.122901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:19.182858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8fe3c5c28964b768b2c0023fc62f163'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.189923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8fe3c5c28964b768b2c0023fc62f163'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.190568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8fe3c5c28964b768b2c0023fc62f163'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.192608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:19.199866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8fe3c5c28964b768b2c0023fc62f163'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.200642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8fe3c5c28964b768b2c0023fc62f163'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.218464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1190/2385] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:11:19 2022] DELETE /identity/v3/users/c8fe3c5c28964b768b2c0023fc62f163 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:11:19.389862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:19.400638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d86b7e-066f-4a0e-a428-8e0453eb0417 service placement] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:19.420523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:19.462976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:19.463940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:19.464225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:19.464495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] PATH_INFO: `/v3/projects/cd3d02b11bdf45fe99687fda3d1078cd` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:19.465633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3d02b11bdf45fe99687fda3d1078cd)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:19.467600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:19.521491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3d02b11bdf45fe99687fda3d1078cd', 'request_id': 'req-711b7e61-48b9-422a-8fb0-e709e74cda49'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.529324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3d02b11bdf45fe99687fda3d1078cd', 'request_id': 'req-711b7e61-48b9-422a-8fb0-e709e74cda49'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.538801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3d02b11bdf45fe99687fda3d1078cd', 'request_id': 'req-711b7e61-48b9-422a-8fb0-e709e74cda49'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.551855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] The token cache is being invalidate because project cd3d02b11bdf45fe99687fda3d1078cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:11:19.552138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.552775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.554292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1196/2386] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:11:19 2022] DELETE /identity/v3/projects/cd3d02b11bdf45fe99687fda3d1078cd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:11:19.558949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:19.560410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:19.573819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50130c6c-3a5e-4abf-b6f4-7d58acb78168 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:19.632700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:19.633460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:19.633741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:19.634030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] PATH_INFO: `/v3/users/8baf517b712c4825ba0ac6c786d43c74` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:19.635159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baf517b712c4825ba0ac6c786d43c74)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:19.640391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:19.710626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baf517b712c4825ba0ac6c786d43c74'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.720710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baf517b712c4825ba0ac6c786d43c74'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.721878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baf517b712c4825ba0ac6c786d43c74'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.731875 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baf517b712c4825ba0ac6c786d43c74'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.732741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baf517b712c4825ba0ac6c786d43c74'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:19.755296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1191/2387] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:11:19 2022] DELETE /identity/v3/users/8baf517b712c4825ba0ac6c786d43c74 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:11:19.832666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:19.835145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:19.926166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b7e61-48b9-422a-8fb0-e709e74cda49 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:19.947596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:19.949208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:19.986829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:20.013389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:20.014193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:20.014469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:20.014736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:20.016650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:20.020387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:20.036434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a31fb1-9220-4ae0-88bc-ec81b031d0cd admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:20.042548 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:20.043187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:20.043463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:20.043735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] PATH_INFO: `/v3/projects/ebc2f3caf6f74c1192a858fc0f9aca5f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:20.045165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc2f3caf6f74c1192a858fc0f9aca5f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:20.047157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:20.106538 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:20.108188 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:20.109755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc2f3caf6f74c1192a858fc0f9aca5f', 'request_id': 'req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:20.117512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc2f3caf6f74c1192a858fc0f9aca5f', 'request_id': 'req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:20.123959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc2f3caf6f74c1192a858fc0f9aca5f', 'request_id': 'req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:20.129600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1197/2388] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:20.140583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] The token cache is being invalidate because project ebc2f3caf6f74c1192a858fc0f9aca5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:11:20.140878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:20.141531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0a8c62f0-d040-41ca-af4d-4f9d4ceadad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:11:20.143047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1192/2389] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:11:19 2022] DELETE /identity/v3/projects/ebc2f3caf6f74c1192a858fc0f9aca5f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:11:22.631812 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:22.633239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:22.693844 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e0814-1b5c-402c-baed-0b5390a12d4e service nova] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:22.742461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:22.743608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:22.743886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:22.744159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:22.745896 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:22.775033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:22.808581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:22.810027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:22.819261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1198/2390] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:26.584148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcf5b7-8a13-4fce-a29d-5ac84841593b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:26.584679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcf5b7-8a13-4fce-a29d-5ac84841593b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:26.584679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dcf5b7-8a13-4fce-a29d-5ac84841593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:26.655604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-78dcf5b7-8a13-4fce-a29d-5ac84841593b None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:11:26.656821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-78dcf5b7-8a13-4fce-a29d-5ac84841593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:26.866788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1193/2391] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:11:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:11:26.877091 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:26.878614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:26.977910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c0a46-9d3c-451f-a1e8-76ce5a8531f8 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:26.983447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:26.984066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:26.984469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:26.984749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:26.997344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:26.999577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:27.010799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1199/2392] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:11:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:11:36.860530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=67,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:37.005260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=57 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:40.051431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:40.051983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:40.051983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:40.079146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] MFA Rules not processed for user `74a8cc19ab204424ab23c62915ac2bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:11:40.080344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:40.133446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1194/2393] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:11:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:11:40.499865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:40.501309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:40.513016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed6122-b4b9-4531-a533-59045a5b1fda service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:40.516602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:40.517191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:40.517467 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:40.517736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:40.519400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:40.532404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:40.534632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:40.546589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1200/2394] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:43.853438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:43.855151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:43.938962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b063ac-dc10-4f5b-8642-492cccaaf839 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:43.942880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:43.943494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:43.943767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:43.944033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:43.945817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:44.019955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:44.021520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:44.034446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1195/2395] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:44.201959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:44.211497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cc4f3-2282-4c03-b702-ef5771e9d4b1 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:44.215339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:44.216027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:44.216309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:44.216579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:44.218356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:44.228782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:44.230776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:44.240024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1201/2396] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:44.271769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:44.273777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:44.292623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd96c-fe8c-4610-adae-a44bd05c7a66 service glance] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:44.298186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:44.298793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:44.299088 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:44.299332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:44.309279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:44.322294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:44.323926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292be7a9-3859-4575-9948-da9780fe5fb1 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:44.336607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1196/2397] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:44.423647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:44.433495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a67c9-8dea-403d-b6c6-781d543e5c16 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:44.437118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:44.437714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:44.437990 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:44.438291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:44.440012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:44.453529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:44.454999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:44.473575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1202/2398] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:11:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:11:49.080168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:49.080652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:49.080841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:49.095898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:11:49.101689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:11:49.102987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:11:49.110101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1197/2399] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:11:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:11:49.122065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:11:49.133798 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecfffe-25f5-49a3-b985-977a9e45d364 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:11:49.139021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ac939-fa41-423e-86ad-48b25e6ac8d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:11:49.139620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ac939-fa41-423e-86ad-48b25e6ac8d0 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:11:49.139909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ac939-fa41-423e-86ad-48b25e6ac8d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:11:49.140189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ac939-fa41-423e-86ad-48b25e6ac8d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:11:49.149085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ac939-fa41-423e-86ad-48b25e6ac8d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:11:49.150721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ac939-fa41-423e-86ad-48b25e6ac8d0 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:11:49.159338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1203/2400] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:11:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:11:59.155474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=24 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:05.683823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:05.685345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:05.700818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19188b70-3b2e-4e49-af92-86c81b95feb7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:05.706185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:05.706784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:05.707063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:05.707329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:05.708994 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:05.798761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:05.800347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:05.819114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1198/2401] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:13.058915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:13.059440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:13.059440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:13.079597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:13.080810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:13.088802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1204/2402] 10.222.0.50 () {58 vars in 967 bytes} [Sat Sep 24 05:12:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:13.101043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:13.112150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54ea01-1157-45f5-885d-14f776ee1f12 service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:13.117110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:13.117714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:13.118007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:13.118293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:13.126767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:13.128357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:13.137362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1199/2403] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:12:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:12:23.074462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:23.134289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=47 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:38.957320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:38.966618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afb33d-f5a5-42eb-b7f2-36aa45171aa0 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:38.970129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:38.970718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:38.971001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:38.971267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:38.972913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:39.036775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:39.038244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:39.048811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1205/2404] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:42.017082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.019163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:42.086044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.089071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:42.107689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef928f-07f3-4d5a-8f03-27b5ca68f8ec service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.108412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19879e7-6729-4146-8880-0f24b6a472d5 service swift] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.112366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.112963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.113287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.113567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] PATH_INFO: `/v3/users/4b1b0448e7f54e3aaae49e411699269e` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.113695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.114369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.114654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.114837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1b0448e7f54e3aaae49e411699269e)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.114956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.116739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:42.116998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.129583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.131220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.141544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1200/2405] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:42.181226 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1b0448e7f54e3aaae49e411699269e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.189624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1b0448e7f54e3aaae49e411699269e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.190348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1b0448e7f54e3aaae49e411699269e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.191022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1b0448e7f54e3aaae49e411699269e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.198183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8171e65a-1278-4ce9-a2cb-f91d7dfd47f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1b0448e7f54e3aaae49e411699269e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.217276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1206/2406] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:42 2022] DELETE /identity/v3/users/4b1b0448e7f54e3aaae49e411699269e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:42.390308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.401252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef321500-0301-41ee-99b7-229cff0b6a9f service placement] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.456204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.456826 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.457103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.457373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] PATH_INFO: `/v3/projects/66ec8bde739145dda504995dbea0914a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.458497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66ec8bde739145dda504995dbea0914a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.460444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.506744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66ec8bde739145dda504995dbea0914a', 'request_id': 'req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.513663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66ec8bde739145dda504995dbea0914a', 'request_id': 'req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.519189 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66ec8bde739145dda504995dbea0914a', 'request_id': 'req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.530277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] The token cache is being invalidate because project 66ec8bde739145dda504995dbea0914a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:42.530558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.531196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-07f0e6d2-3921-4f3d-ae2e-2fba39e39155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:42.532889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1201/2407] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:42 2022] DELETE /identity/v3/projects/66ec8bde739145dda504995dbea0914a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:42.537740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.538043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.538318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.559682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:42.560871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:42.623024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1207/2408] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:42.631006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.631293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.631565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.654789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:42.656102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:42.666775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1202/2409] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:42.671393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.681393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982d868-0183-4145-b0e9-1477ca617f9e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.686181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.686782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.687050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.687426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.688366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:42.688366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.689821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.691462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:42.707869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1208/2410] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:12:42.709026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.718864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3330e-ac3b-44d1-a6ea-9c3919609e3c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.723549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.724145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.724426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.724696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.725317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.727741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.742081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:42.772273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1203/2411] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:42 2022] POST /identity/v3/projects => generated 405 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:42.777453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.790924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96069a-dc76-4b5e-a3d5-b3e2a2ab8c11 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.851020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.851648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.851964 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.852246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.852877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.854817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.881213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1209/2412] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:42 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:42.882522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.893286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c0c9-76ba-4f47-bea5-dd731ffe2214 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.897824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.898457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.898728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.899658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.900291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.901868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.912296 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1204/2413] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:42.916448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.926601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf614794-4386-4879-8fb8-b3da9c2f327e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.931166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec061e1-e4e5-4363-89b4-b6bc8a6d8b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.931771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec061e1-e4e5-4363-89b4-b6bc8a6d8b44 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.932035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec061e1-e4e5-4363-89b4-b6bc8a6d8b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.932304 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec061e1-e4e5-4363-89b4-b6bc8a6d8b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.932934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec061e1-e4e5-4363-89b4-b6bc8a6d8b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.934612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec061e1-e4e5-4363-89b4-b6bc8a6d8b44 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.944968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1210/2414] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:42.949732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:42.960103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851a82-2c3c-4d42-aa50-be47f5ebf356 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:42.966045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:42.966719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:42.966998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:42.967271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] PATH_INFO: `/v3/projects/265a4465d3594ab6a6aab4a10163ae4f/users/d409bcb65c2b4533842fe6c8823e0bcf/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:42.982172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=265a4465d3594ab6a6aab4a10163ae4f, user_id=d409bcb65c2b4533842fe6c8823e0bcf, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:42.984822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:42.997601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1205/2415] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:42 2022] PUT /identity/v3/projects/265a4465d3594ab6a6aab4a10163ae4f/users/d409bcb65c2b4533842fe6c8823e0bcf/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:43.008649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.008912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.009179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.037567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] MFA Rules not processed for user `d409bcb65c2b4533842fe6c8823e0bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:43.038835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:43.101714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1211/2416] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:43 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:43.106267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.166402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea2f3c-21a1-4aa4-aca3-bfe2de748557 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.170885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.171476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.171750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.172012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.172623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.174514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.201460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1206/2417] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:43 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:43.206166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.216880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdecdf-f8ca-457f-965c-65579f8c4cc8 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.271949 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.272537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.272808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.273077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.273702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.275823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.284664 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:43.297501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1212/2418] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:43 2022] POST /identity/v3/users => generated 309 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:12:43.301561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.312092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f467bd-d010-47b8-9d7b-7dee1a2d12af admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.316584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.317176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.317449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.317714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.318456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.320025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.330799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1207/2419] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:43.334882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.345090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006235ff-9350-4691-a3b9-4a2e3ce17921 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.349689 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.350372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.350659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.350953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] PATH_INFO: `/v3/projects/e9869e0cc71b40528a8b45f48144d77b/users/9d23d135f85e40879879375bf046fb9a/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.365400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9869e0cc71b40528a8b45f48144d77b, user_id=9d23d135f85e40879879375bf046fb9a, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.368572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.380865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1213/2420] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:43 2022] PUT /identity/v3/projects/e9869e0cc71b40528a8b45f48144d77b/users/9d23d135f85e40879879375bf046fb9a/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:43.385771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.395646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f31443-4daa-47a2-bdb4-7ba7210d0cde admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.449895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eca73d-46cf-46e8-bf60-71264b529c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.450514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eca73d-46cf-46e8-bf60-71264b529c55 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.450788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eca73d-46cf-46e8-bf60-71264b529c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.451054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eca73d-46cf-46e8-bf60-71264b529c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.451672 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eca73d-46cf-46e8-bf60-71264b529c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.453403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eca73d-46cf-46e8-bf60-71264b529c55 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.463578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1208/2421] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:43.467839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.478662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748590ec-3e1c-4956-aeeb-ea39e0a00697 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.483318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.483961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.484240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.484514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] PATH_INFO: `/v3/projects/e9869e0cc71b40528a8b45f48144d77b/users/9d23d135f85e40879879375bf046fb9a/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.486435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9869e0cc71b40528a8b45f48144d77b, user_id=9d23d135f85e40879879375bf046fb9a, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.489096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.499623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1214/2422] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:43 2022] PUT /identity/v3/projects/e9869e0cc71b40528a8b45f48144d77b/users/9d23d135f85e40879879375bf046fb9a/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:43.516413 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.516703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.516994 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.544517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] MFA Rules not processed for user `9d23d135f85e40879879375bf046fb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:43.545704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:43.550937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:43.613734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1209/2423] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:43 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:43.625182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.626895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:43.684235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1ebf34-90a7-4a46-84e0-36becad62d09 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.693113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c6cc-b832-4755-b22f-fd7097c4f844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.693740 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c6cc-b832-4755-b22f-fd7097c4f844 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.694047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c6cc-b832-4755-b22f-fd7097c4f844 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.694326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c6cc-b832-4755-b22f-fd7097c4f844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.703704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c6cc-b832-4755-b22f-fd7097c4f844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.705127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62c6cc-b832-4755-b22f-fd7097c4f844 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.714506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1215/2424] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:43 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:43.848157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:43.908451 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4046723-739c-43cf-b358-5be2c0d2afbd None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:43.912925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:43.913547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:43.913823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:43.914129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] PATH_INFO: `/v3/users/d409bcb65c2b4533842fe6c8823e0bcf` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:43.915829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d409bcb65c2b4533842fe6c8823e0bcf)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:43.917725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:43.963849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd409bcb65c2b4533842fe6c8823e0bcf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:43.971579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd409bcb65c2b4533842fe6c8823e0bcf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:43.972632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd409bcb65c2b4533842fe6c8823e0bcf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:43.980689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd409bcb65c2b4533842fe6c8823e0bcf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:43.981347 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd409bcb65c2b4533842fe6c8823e0bcf'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.000449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1210/2425] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:43 2022] DELETE /identity/v3/users/d409bcb65c2b4533842fe6c8823e0bcf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:44.005157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:44.005443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:44.005721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:44.028907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:44.030162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:44.091640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1216/2426] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:44.101208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:44.103289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:44.130288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:44.163054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a213b9-034c-4226-aa50-46f0a4851c6e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:44.167323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:44.167921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:44.168200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:44.168473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:44.177235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:44.178701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:44.187921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1211/2427] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:44.460993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:44.463539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:44.476609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e621-9da3-4e27-8031-14096f0f6560 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:44.481439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:44.482184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:44.482630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:44.483078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] PATH_INFO: `/v3/projects/265a4465d3594ab6a6aab4a10163ae4f` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:44.484353 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=265a4465d3594ab6a6aab4a10163ae4f)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:44.486810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:44.534195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265a4465d3594ab6a6aab4a10163ae4f', 'request_id': 'req-3982b23a-ac14-40a9-951d-2bc58e6e52ff'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.536102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:44.540875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265a4465d3594ab6a6aab4a10163ae4f', 'request_id': 'req-3982b23a-ac14-40a9-951d-2bc58e6e52ff'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.548749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265a4465d3594ab6a6aab4a10163ae4f', 'request_id': 'req-3982b23a-ac14-40a9-951d-2bc58e6e52ff'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.560109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] The token cache is being invalidate because project 265a4465d3594ab6a6aab4a10163ae4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:44.560398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.561026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.562551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1217/2428] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:44 2022] DELETE /identity/v3/projects/265a4465d3594ab6a6aab4a10163ae4f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:44.566529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:44.576888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cff14e-e226-480e-a84e-72a316da21ce service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:44.639030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:44.639838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:44.640135 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:44.640403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] PATH_INFO: `/v3/users/9d23d135f85e40879879375bf046fb9a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:44.641535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d23d135f85e40879879375bf046fb9a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:44.643666 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:44.700076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d23d135f85e40879879375bf046fb9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.707145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d23d135f85e40879879375bf046fb9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.707763 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d23d135f85e40879879375bf046fb9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.714723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d23d135f85e40879879375bf046fb9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.715382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-6eeee042-307b-448c-8a47-78f53b9cb834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d23d135f85e40879879375bf046fb9a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:44.732170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1212/2429] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:44 2022] DELETE /identity/v3/users/9d23d135f85e40879879375bf046fb9a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:45.010662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.012180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:45.025760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982b23a-ac14-40a9-951d-2bc58e6e52ff admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.081059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.081703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.081966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.082268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] PATH_INFO: `/v3/projects/e9869e0cc71b40528a8b45f48144d77b` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.083378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9869e0cc71b40528a8b45f48144d77b)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.085298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.131002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9869e0cc71b40528a8b45f48144d77b', 'request_id': 'req-3dd10279-d3e9-48fe-889c-bf36378a2fd5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:45.137251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9869e0cc71b40528a8b45f48144d77b', 'request_id': 'req-3dd10279-d3e9-48fe-889c-bf36378a2fd5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:45.144762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9869e0cc71b40528a8b45f48144d77b', 'request_id': 'req-3dd10279-d3e9-48fe-889c-bf36378a2fd5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:45.156885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] The token cache is being invalidate because project e9869e0cc71b40528a8b45f48144d77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:45.157426 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:45.158097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-3dd10279-d3e9-48fe-889c-bf36378a2fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:45.159574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1218/2430] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:45 2022] DELETE /identity/v3/projects/e9869e0cc71b40528a8b45f48144d77b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:45.164673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.165149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.165266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.185783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:45.186935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:45.228145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:45.249038 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1213/2431] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:45.256993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.257301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.257574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.278968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:45.280412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:45.288503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1219/2432] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:45.292756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.303079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc51b-d855-43a3-a0c6-dc44ad2bdd1a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.308561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.309162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.309435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.309702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.310384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:45.310680 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.312486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.324632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1214/2433] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:12:45.328847 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.337897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77c80b-6f49-431a-94dc-ac7002b4ec21 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.342474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.343061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.343335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.343604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.344218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.346126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.372571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1220/2434] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:45.376627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.386858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f5f4d-efea-439a-9ac2-c74a73d87034 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.440725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.441252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.441528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.441823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.442481 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.444405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.465212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1215/2435] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:12:45.469319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.480023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4be832-619d-4414-b429-0f85b090322f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.484549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.485135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.485406 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.485672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.486327 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.487904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.498078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1221/2436] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:45.501872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.512087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067ad16-559e-4954-8ace-85b79765c318 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.516582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dd806-e20e-472b-8644-38c42b4bde11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.517164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd806-e20e-472b-8644-38c42b4bde11 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.517442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd806-e20e-472b-8644-38c42b4bde11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.517706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd806-e20e-472b-8644-38c42b4bde11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.518394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dd806-e20e-472b-8644-38c42b4bde11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.519961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dd806-e20e-472b-8644-38c42b4bde11 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.530162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1216/2437] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:12:45.534181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.543873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf410862-a302-48f7-8faa-5b26b4a90fac admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.548536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.549172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.549447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.549715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] PATH_INFO: `/v3/projects/b3fe8a830ab742d3b3512efa8791577e/users/74a6fe38e9924c8f8146455bc232933e/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.564918 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fe8a830ab742d3b3512efa8791577e, user_id=74a6fe38e9924c8f8146455bc232933e, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.567590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.580041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1222/2438] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:45 2022] PUT /identity/v3/projects/b3fe8a830ab742d3b3512efa8791577e/users/74a6fe38e9924c8f8146455bc232933e/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:45.590144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.590429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.590700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.617018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] MFA Rules not processed for user `74a6fe38e9924c8f8146455bc232933e`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:45.618757 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:45.683354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1217/2439] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:45.687888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.696946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:45.749823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e37f25-3845-4680-b68d-8d048af19736 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.754471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.755064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.755342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.755615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.756232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.758159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.784832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1223/2440] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:45.788927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.801670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7caa4-71a0-4048-945d-08ecbcd37106 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.838210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:45.856683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.857284 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.857560 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.857830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.858509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.860410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.880789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1218/2441] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:45 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:45.884857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.894923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f38512-2484-4d33-a8cd-a9b6bfd08994 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.899544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.900135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.900413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.900683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.901313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.902936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.913495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1224/2442] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:45.918035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.927608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a6e3c-3020-42ff-b065-f6af185f6730 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:45.932636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:45.933267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:45.933542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:45.933808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] PATH_INFO: `/v3/projects/790c7483cfb14878aeceb1601d0baab4/users/f43de013b2ce4c439f7659cc6fd9be80/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:45.948006 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=790c7483cfb14878aeceb1601d0baab4, user_id=f43de013b2ce4c439f7659cc6fd9be80, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:45.950646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:45.962443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1219/2443] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:45 2022] PUT /identity/v3/projects/790c7483cfb14878aeceb1601d0baab4/users/f43de013b2ce4c439f7659cc6fd9be80/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:45.966418 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:45.976245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b654f0f8-8b9b-498b-8ce0-4f31362cf021 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:46.031055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa69a5-a19d-4b3d-89bd-c334ac8d756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:46.031656 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa69a5-a19d-4b3d-89bd-c334ac8d756d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:46.031935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa69a5-a19d-4b3d-89bd-c334ac8d756d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:46.032463 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa69a5-a19d-4b3d-89bd-c334ac8d756d admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:46.033094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa69a5-a19d-4b3d-89bd-c334ac8d756d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:46.034719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa69a5-a19d-4b3d-89bd-c334ac8d756d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:46.044579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1225/2444] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:46.048705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:46.059465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee8640-2461-4342-9fe0-6a2589e20565 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:46.064011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:46.064653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:46.064925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:46.065196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] PATH_INFO: `/v3/projects/790c7483cfb14878aeceb1601d0baab4/users/f43de013b2ce4c439f7659cc6fd9be80/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:46.067094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] RBAC: Authorizing `identity:create_grant(project_id=790c7483cfb14878aeceb1601d0baab4, user_id=f43de013b2ce4c439f7659cc6fd9be80, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:46.069703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:46.079606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1220/2445] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:46 2022] PUT /identity/v3/projects/790c7483cfb14878aeceb1601d0baab4/users/f43de013b2ce4c439f7659cc6fd9be80/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:46.091388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:46.091674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:46.091946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:46.119392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] MFA Rules not processed for user `f43de013b2ce4c439f7659cc6fd9be80`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:46.120590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:46.186810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1226/2446] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:46.199001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:46.200643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:46.256350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4da12c-cdb9-4be0-936f-c7a2a156272f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:46.260501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:46.261090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:46.261366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:46.261661 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:46.270352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:46.271768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:46.280924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1221/2447] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:46.326901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:46.328490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:46.335669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:46.342508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3ab45-9c3a-46f4-9f94-c7f59bbf4fae None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:46.347461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a819bc-8066-4880-9bb4-a89442d47efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:46.348059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a819bc-8066-4880-9bb4-a89442d47efe service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:46.348333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a819bc-8066-4880-9bb4-a89442d47efe service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:46.348600 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a819bc-8066-4880-9bb4-a89442d47efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:46.405590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a819bc-8066-4880-9bb4-a89442d47efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:46.407058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a819bc-8066-4880-9bb4-a89442d47efe service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:46.417074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1227/2448] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:49.698414 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:49.758190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df483875-bd14-46bc-958d-bb0cb6c605b2 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:49.763447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:49.764095 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:49.764368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:49.764635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] PATH_INFO: `/v3/users/74a6fe38e9924c8f8146455bc232933e` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:49.765732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a6fe38e9924c8f8146455bc232933e)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:49.767723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:49.804050 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:49.814355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a6fe38e9924c8f8146455bc232933e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:49.822032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a6fe38e9924c8f8146455bc232933e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:49.822636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a6fe38e9924c8f8146455bc232933e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:49.829567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a6fe38e9924c8f8146455bc232933e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:49.830236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a6fe38e9924c8f8146455bc232933e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:49.846858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1222/2449] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:49 2022] DELETE /identity/v3/users/74a6fe38e9924c8f8146455bc232933e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:49.851495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:49.851946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:49.852042 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:49.872809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:49.874025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:49.932128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1228/2450] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:49.959783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:49.961967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:50.030936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd58ba-b122-415b-90d1-e31839cf1db0 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:50.034894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:50.035509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:50.035787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:50.036059 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:50.045422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:50.046898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:50.057213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1223/2451] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:12:50.323013 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:50.324429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:50.336919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5e138-fd68-47b7-a17d-45827b4ce094 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:50.343148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:50.343787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:50.344060 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:50.344341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] PATH_INFO: `/v3/projects/b3fe8a830ab742d3b3512efa8791577e` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:50.345436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fe8a830ab742d3b3512efa8791577e)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:50.347508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:50.393869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fe8a830ab742d3b3512efa8791577e', 'request_id': 'req-1a0de3b6-846b-43c2-9a35-90081c26e410'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.399369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fe8a830ab742d3b3512efa8791577e', 'request_id': 'req-1a0de3b6-846b-43c2-9a35-90081c26e410'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.407552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fe8a830ab742d3b3512efa8791577e', 'request_id': 'req-1a0de3b6-846b-43c2-9a35-90081c26e410'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.418830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] The token cache is being invalidate because project b3fe8a830ab742d3b3512efa8791577e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:50.419121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.419745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.421199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1229/2452] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:50 2022] DELETE /identity/v3/projects/b3fe8a830ab742d3b3512efa8791577e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:12:50.425144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:50.435595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991842ea-d8af-40da-a7d6-bb7a46c807f8 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:50.491027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:50.491650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:50.491922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:50.492193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] PATH_INFO: `/v3/users/f43de013b2ce4c439f7659cc6fd9be80` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:50.493898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43de013b2ce4c439f7659cc6fd9be80)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:50.495834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:50.541211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43de013b2ce4c439f7659cc6fd9be80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.548116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43de013b2ce4c439f7659cc6fd9be80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.548891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43de013b2ce4c439f7659cc6fd9be80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.553427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:50.556994 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43de013b2ce4c439f7659cc6fd9be80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.557766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-e482cc4f-b493-4b65-b1ba-e6cf12c48c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43de013b2ce4c439f7659cc6fd9be80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.576404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1224/2453] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:50 2022] DELETE /identity/v3/users/f43de013b2ce4c439f7659cc6fd9be80 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:50.863710 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:50.865245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:50.883411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0de3b6-846b-43c2-9a35-90081c26e410 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:50.904433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:50.942421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:50.943058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:50.943335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:50.943607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] PATH_INFO: `/v3/projects/790c7483cfb14878aeceb1601d0baab4` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:50.944714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=790c7483cfb14878aeceb1601d0baab4)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:50.947176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:50.992163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '790c7483cfb14878aeceb1601d0baab4', 'request_id': 'req-a641e32e-9414-4d66-b5cd-fb3209fba646'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:50.997705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '790c7483cfb14878aeceb1601d0baab4', 'request_id': 'req-a641e32e-9414-4d66-b5cd-fb3209fba646'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:51.004105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '790c7483cfb14878aeceb1601d0baab4', 'request_id': 'req-a641e32e-9414-4d66-b5cd-fb3209fba646'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:51.015505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] The token cache is being invalidate because project 790c7483cfb14878aeceb1601d0baab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:51.015786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:51.016402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a641e32e-9414-4d66-b5cd-fb3209fba646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:51.017836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1230/2454] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:50 2022] DELETE /identity/v3/projects/790c7483cfb14878aeceb1601d0baab4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:12:51.022762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.023232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.023350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.043357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:51.044550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:51.103534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1225/2455] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:51.111520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.111801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.113027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.135737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:51.136992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:51.145061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1231/2456] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:51.150214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.160513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983088be-9512-4108-ac76-4bacd75145b1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.165479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.166146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.166427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.166714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.167341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:51.167640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.169231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.184957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1226/2457] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:12:51.189771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.200413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed7f32-4a86-4d34-bd82-cbf28e730f18 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.205822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.206036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.206390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.206734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.207429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.209413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.239638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1232/2458] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:51.244012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.254392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3430d-0430-4132-845b-ffebe4b0ad43 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.311216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.311823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.312105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.312380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.313002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.314952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.335830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1227/2459] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:51.340081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.350234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7636433-cadd-47dd-9a27-ad5ff73c9178 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.354843 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.355431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.355709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.355977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.356607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.358217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.369362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1233/2460] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:12:51.373167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.383032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cb829-e39c-4ce2-8008-0ea485ee3867 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.388191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac9ee9-5931-4c4f-9aea-9704338a6d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.388820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9ee9-5931-4c4f-9aea-9704338a6d47 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.389084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9ee9-5931-4c4f-9aea-9704338a6d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.389354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac9ee9-5931-4c4f-9aea-9704338a6d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.390008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac9ee9-5931-4c4f-9aea-9704338a6d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.391606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac9ee9-5931-4c4f-9aea-9704338a6d47 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.402223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1228/2461] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:51.406102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.416073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1637f53-2335-4a07-b8d4-9af4d12d283f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.420599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.421226 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.421509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.421779 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] PATH_INFO: `/v3/projects/d571074946de43cfa2d6f89b11a6f8e3/users/1e1b11d6e1ee45578f629a9856721d1e/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.436231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d571074946de43cfa2d6f89b11a6f8e3, user_id=1e1b11d6e1ee45578f629a9856721d1e, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.438917 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.451894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1234/2462] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:51 2022] PUT /identity/v3/projects/d571074946de43cfa2d6f89b11a6f8e3/users/1e1b11d6e1ee45578f629a9856721d1e/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:51.463354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.463837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.464062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.491011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] MFA Rules not processed for user `1e1b11d6e1ee45578f629a9856721d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:51.492221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:51.521673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:51.556741 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1229/2463] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:12:51.561217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.610259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:51.622756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d8340-3cc3-4838-aed6-5e84c072327c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.627823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.628425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.628706 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.628977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.629627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.631576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.660447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1235/2464] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:51.665492 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.676015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839c895-e50b-4e74-9f64-51c2ccdc8aea None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.731139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.731739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.732018 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.732289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.732921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.734970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.755653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1230/2465] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:51.760152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.770390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bccfb3-6fd0-43d2-9c3a-21cf7ea1738b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.775667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.776203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.776479 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.776752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.777384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.779003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.789604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1236/2466] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:51.793794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.803497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c23a-4c42-47e0-a8b7-54c8ac194ccd admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.808012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.808643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.808919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.809190 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] PATH_INFO: `/v3/projects/165f860ca6464585ab0aafe1241c2960/users/c78be45bc37847d8a89511fb4d7864bb/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.824923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=165f860ca6464585ab0aafe1241c2960, user_id=c78be45bc37847d8a89511fb4d7864bb, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.827578 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.840712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1231/2467] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:51 2022] PUT /identity/v3/projects/165f860ca6464585ab0aafe1241c2960/users/c78be45bc37847d8a89511fb4d7864bb/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:51.844632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.854473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da11c80-35a9-4b6e-af4e-079a4ec10599 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.911119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ff545-1eff-4d51-95fd-7d01d91e0bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.911807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff545-1eff-4d51-95fd-7d01d91e0bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.911991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff545-1eff-4d51-95fd-7d01d91e0bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.912264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ff545-1eff-4d51-95fd-7d01d91e0bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.912901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ff545-1eff-4d51-95fd-7d01d91e0bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.914558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ff545-1eff-4d51-95fd-7d01d91e0bc8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.924109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1237/2468] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:51.928275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:51.938194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeba1a9-66cf-44af-8f2d-7d05ffbcb45c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:51.942735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:51.943375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.943653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.943926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] PATH_INFO: `/v3/projects/165f860ca6464585ab0aafe1241c2960/users/c78be45bc37847d8a89511fb4d7864bb/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.946326 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165f860ca6464585ab0aafe1241c2960, user_id=c78be45bc37847d8a89511fb4d7864bb, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:51.948929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:51.958888 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1232/2469] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:51 2022] PUT /identity/v3/projects/165f860ca6464585ab0aafe1241c2960/users/c78be45bc37847d8a89511fb4d7864bb/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:51.969486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:51.969773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:51.970073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:51.997505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] MFA Rules not processed for user `c78be45bc37847d8a89511fb4d7864bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:51.998736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:52.065338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1238/2470] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:52.078170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:52.079836 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:52.135748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2411007-a2a3-4523-b9c4-0c90a9213698 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:52.139322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:52.139906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:52.140183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:52.140447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:52.149098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:52.150553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:52.160710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1233/2471] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:52.479711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:52.488857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187a51c-78c8-41eb-8044-3d5f80f9b17e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:52.494144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c78be45bc37847d8a89511fb4d7864bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '165f860ca6464585ab0aafe1241c2960', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:52.494742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:52.495023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:52.495297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:52.495966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:52.496268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:52.497859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:52.509154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1239/2472] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:12:52.513206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:52.522121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3e79-3fc6-44f9-8c2f-e66c1f762f92 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:52.526660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c78be45bc37847d8a89511fb4d7864bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '165f860ca6464585ab0aafe1241c2960', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:52.527247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:52.527529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:52.527804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:52.528428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:52.530348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:52.557369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1234/2473] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:52 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:52.561413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:52.571537 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e77cd-c429-4eaa-9617-84e48768f45e tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:52.600044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:52.627565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c78be45bc37847d8a89511fb4d7864bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '165f860ca6464585ab0aafe1241c2960', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:52.628106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:52.628387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:52.628659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:52.629298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:52.629597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:52.631275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:52.643251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1240/2474] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:12:52.681604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:52.688089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:52.692479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8fb5d-f618-4487-ae23-b46996bbbbb0 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:52.698609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c78be45bc37847d8a89511fb4d7864bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '165f860ca6464585ab0aafe1241c2960', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:52.699285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:52.699604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:52.699926 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] PATH_INFO: `/v3/projects/bdcad79edde44028b88ca9b72c0d87d3` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:52.701141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bdcad79edde44028b88ca9b72c0d87d3)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:52.703177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:52.748447 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdcad79edde44028b88ca9b72c0d87d3', 'request_id': 'req-5d011598-50d6-4238-80af-7f2824359bcd'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:52.755676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdcad79edde44028b88ca9b72c0d87d3', 'request_id': 'req-5d011598-50d6-4238-80af-7f2824359bcd'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:52.761290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdcad79edde44028b88ca9b72c0d87d3', 'request_id': 'req-5d011598-50d6-4238-80af-7f2824359bcd'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:52.772464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] The token cache is being invalidate because project bdcad79edde44028b88ca9b72c0d87d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:52.772760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:52.773382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5d011598-50d6-4238-80af-7f2824359bcd tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:52.775344 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1235/2475] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:52 2022] DELETE /identity/v3/projects/bdcad79edde44028b88ca9b72c0d87d3 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:52.963331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:52.965238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:53.028366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b8394-dd6a-4e1c-a3c1-41bc5161dc50 tempest-VolumeQuotaClassesTest-1806086621 tempest-VolumeQuotaClassesTest-1806086621-project-admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:53.087764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:53.088394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:53.088672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:53.088950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] PATH_INFO: `/v3/users/1e1b11d6e1ee45578f629a9856721d1e` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:53.090073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1b11d6e1ee45578f629a9856721d1e)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:53.091981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:53.136502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1b11d6e1ee45578f629a9856721d1e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.143620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1b11d6e1ee45578f629a9856721d1e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.144239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1b11d6e1ee45578f629a9856721d1e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.144833 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1b11d6e1ee45578f629a9856721d1e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.152111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1b11d6e1ee45578f629a9856721d1e'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.162113 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:53.172166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1241/2476] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:52 2022] DELETE /identity/v3/users/1e1b11d6e1ee45578f629a9856721d1e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:53.176595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:53.177124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:53.177245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:53.195505 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:53.197223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:53.255735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1236/2477] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:53.266125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:53.267487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:53.324455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee8a2d-0a12-44d3-8eee-8e3e57a99a5b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:53.328086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:53.328684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:53.328959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:53.329229 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:53.338151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:53.339570 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:53.348759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1242/2478] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:53.713704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:53.715401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:53.728683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27e6dc1-bab5-4494-881a-df7719d91e4f None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:53.733362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:53.734003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:53.734301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:53.734571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] PATH_INFO: `/v3/projects/d571074946de43cfa2d6f89b11a6f8e3` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:53.735684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d571074946de43cfa2d6f89b11a6f8e3)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:53.737759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:53.793080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd571074946de43cfa2d6f89b11a6f8e3', 'request_id': 'req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.800437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd571074946de43cfa2d6f89b11a6f8e3', 'request_id': 'req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.806316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd571074946de43cfa2d6f89b11a6f8e3', 'request_id': 'req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.817167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] The token cache is being invalidate because project d571074946de43cfa2d6f89b11a6f8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:53.817452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.818097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.819556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1237/2479] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:53 2022] DELETE /identity/v3/projects/d571074946de43cfa2d6f89b11a6f8e3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:53.823805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:53.833209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070f6e2-f852-4bd6-b5aa-a2e96541c65e service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:53.887882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:53.888504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:53.888778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:53.889048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] PATH_INFO: `/v3/users/c78be45bc37847d8a89511fb4d7864bb` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:53.890205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78be45bc37847d8a89511fb4d7864bb)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:53.892101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:53.936930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78be45bc37847d8a89511fb4d7864bb'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.944116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78be45bc37847d8a89511fb4d7864bb'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.944732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78be45bc37847d8a89511fb4d7864bb'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.945336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78be45bc37847d8a89511fb4d7864bb'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.952699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b0e12196-0013-452e-add5-7fa45f8646e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78be45bc37847d8a89511fb4d7864bb'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:53.970168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1243/2480] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:53 2022] DELETE /identity/v3/users/c78be45bc37847d8a89511fb4d7864bb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:54.238614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.240083 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:54.254011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c65fc5-6a89-4675-8d98-c0893ecea0ef admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.265597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:54.316476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.317107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.317386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.317659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] PATH_INFO: `/v3/projects/165f860ca6464585ab0aafe1241c2960` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.318811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] RBAC: Authorizing `identity:delete_project(project_id=165f860ca6464585ab0aafe1241c2960)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.320711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.365730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165f860ca6464585ab0aafe1241c2960', 'request_id': 'req-66a53b51-c741-4de4-b147-89869c6aa16f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:54.372715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165f860ca6464585ab0aafe1241c2960', 'request_id': 'req-66a53b51-c741-4de4-b147-89869c6aa16f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:54.378126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165f860ca6464585ab0aafe1241c2960', 'request_id': 'req-66a53b51-c741-4de4-b147-89869c6aa16f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:54.388804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] The token cache is being invalidate because project 165f860ca6464585ab0aafe1241c2960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:54.389090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:54.389710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-66a53b51-c741-4de4-b147-89869c6aa16f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:54.391161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1238/2481] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:54 2022] DELETE /identity/v3/projects/165f860ca6464585ab0aafe1241c2960 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:54.396201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.396556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.396831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.417207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:54.418425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:54.427883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:54.477846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1244/2482] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:54.485755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.486015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.486371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.505371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:54.506606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:54.514366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1239/2483] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:12:54.518552 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.528547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01da00-2c2b-404a-96a0-58ccd076c790 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.533309 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.533899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.534217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.534489 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.535132 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:54.535431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.537004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.548765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1245/2484] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:12:54.552769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.561703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f14b6e-100d-4fc9-91e9-e6e95bc8ca38 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.566260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.566839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.567114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.567382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.568013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.569915 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.597642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1240/2485] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:54 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:54.601379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.612181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bbf79-bd24-4fcb-9183-95b53a2cceb8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.669264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.669876 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.670185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.670461 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.671088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.673019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.693590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1246/2486] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:54 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:54.698323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.708521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10994190-3ccc-4fdc-a462-33ac92492c23 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.713773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.714396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.714669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.714936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.715563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.717143 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.726753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1241/2487] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:54.730480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.740250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea39711f-d0bd-45cd-9754-7469166c64df admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.744817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8808b673-e220-41ca-857c-08709897d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.745401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808b673-e220-41ca-857c-08709897d027 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.745677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808b673-e220-41ca-857c-08709897d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.745945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808b673-e220-41ca-857c-08709897d027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.746611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808b673-e220-41ca-857c-08709897d027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.748216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8808b673-e220-41ca-857c-08709897d027 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.760256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1247/2488] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:54.764196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.774616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2215b-622f-4e3e-b4ed-424168f52785 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.779164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.779837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.780118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.780390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] PATH_INFO: `/v3/projects/5473b885d507428fa06011a96ffcf104/users/d91201d53c0c448b9c90ce6020cbb4f5/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.796118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5473b885d507428fa06011a96ffcf104, user_id=d91201d53c0c448b9c90ce6020cbb4f5, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.798793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:54.818008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1242/2489] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:54 2022] PUT /identity/v3/projects/5473b885d507428fa06011a96ffcf104/users/d91201d53c0c448b9c90ce6020cbb4f5/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:12:54.832462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.832747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.833021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.860577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] MFA Rules not processed for user `d91201d53c0c448b9c90ce6020cbb4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:54.861809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:54.925000 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1248/2490] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:54.929410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:54.969311 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:54.990936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331cc19-1b00-44d7-ba3f-3be99803466c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:54.995386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:54.995980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:54.996260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:54.996530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:54.997158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:54.999150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.024987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1243/2491] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:54 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:55.029112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.040007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738f465-43a9-47a6-ac4f-a89c4e16a28b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.047728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:55.113941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.114577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.114848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.115122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.115759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.118166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.138716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1249/2492] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:55 2022] POST /identity/v3/users => generated 298 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:55.142841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.153007 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d504f3-8bfb-4822-890f-61986190f421 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.157815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.158459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.158739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.159010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.159642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.161235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.171776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1244/2493] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:55.175903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.186496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e6082-ac07-494b-a84c-5894bf4ae4d8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.191137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.191780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.192059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.192330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] PATH_INFO: `/v3/projects/fecd97f748e54675a9ef33aefcddee98/users/a22e242ad15e44639e3ecf313b4b6537/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.207312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecd97f748e54675a9ef33aefcddee98, user_id=a22e242ad15e44639e3ecf313b4b6537, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.209925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.222884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1250/2494] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:55 2022] PUT /identity/v3/projects/fecd97f748e54675a9ef33aefcddee98/users/a22e242ad15e44639e3ecf313b4b6537/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:12:55.226809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.236777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2595b8c-437a-4883-bd92-964e26c8a05a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.294921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da251749-32d5-4460-b013-8f461b180bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.295611 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251749-32d5-4460-b013-8f461b180bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.295734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251749-32d5-4460-b013-8f461b180bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.295944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251749-32d5-4460-b013-8f461b180bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.296567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da251749-32d5-4460-b013-8f461b180bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.298172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da251749-32d5-4460-b013-8f461b180bfa admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.308151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1245/2495] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:12:55.312376 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.323283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa603955-6e77-4223-bc5d-e0ff3e321a37 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.327870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.328511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.328786 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.329056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] PATH_INFO: `/v3/projects/fecd97f748e54675a9ef33aefcddee98/users/a22e242ad15e44639e3ecf313b4b6537/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.330989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecd97f748e54675a9ef33aefcddee98, user_id=a22e242ad15e44639e3ecf313b4b6537, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.333592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.343510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1251/2496] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:12:55 2022] PUT /identity/v3/projects/fecd97f748e54675a9ef33aefcddee98/users/a22e242ad15e44639e3ecf313b4b6537/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:55.353394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.353677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.353948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.381937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] MFA Rules not processed for user `a22e242ad15e44639e3ecf313b4b6537`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:55.383186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:55.451035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1246/2497] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:12:55.461692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.463813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:55.519794 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ce2a2-50ec-40fd-8eb0-4b44f0117e3a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.523393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.523981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.524254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.524518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.533439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.534875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.544644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1252/2498] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:55.645029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:55.654853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c4cc5-4cf8-43a3-9ae1-5f85919933ae None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:55.658643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01878c-c76d-49ff-8af4-2e689b434972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:55.659240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01878c-c76d-49ff-8af4-2e689b434972 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:55.659515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01878c-c76d-49ff-8af4-2e689b434972 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:55.659783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01878c-c76d-49ff-8af4-2e689b434972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:55.690222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:55.717784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01878c-c76d-49ff-8af4-2e689b434972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:55.719261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01878c-c76d-49ff-8af4-2e689b434972 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:55.728534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1247/2499] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:58.446571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:58.506111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcb411-d279-477f-87d5-d8fb4d4d5682 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:58.511050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:58.511666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:58.511942 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:58.512215 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] PATH_INFO: `/v3/users/d91201d53c0c448b9c90ce6020cbb4f5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:58.513317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d91201d53c0c448b9c90ce6020cbb4f5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:58.515243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:58.520469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:58.560260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd91201d53c0c448b9c90ce6020cbb4f5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:58.567333 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd91201d53c0c448b9c90ce6020cbb4f5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:58.567991 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd91201d53c0c448b9c90ce6020cbb4f5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:58.568608 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd91201d53c0c448b9c90ce6020cbb4f5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:58.576059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd91201d53c0c448b9c90ce6020cbb4f5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:58.593369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1253/2500] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:58 2022] DELETE /identity/v3/users/d91201d53c0c448b9c90ce6020cbb4f5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:58.597885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:58.598371 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:58.598487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:58.616891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:58.618137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:58.680934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1248/2501] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:58.690550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:58.691925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:58.747336 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca2f7-591d-487a-ad8a-a5038fb031a3 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:58.750889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:58.751481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:58.751754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:58.752021 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:58.761227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:58.762663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:58.771780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1254/2502] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:12:59.032844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:59.034312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:59.046453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498efd42-facc-4924-80c1-fd0d1206d21c None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:59.051176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:59.051809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.052086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.052356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] PATH_INFO: `/v3/projects/5473b885d507428fa06011a96ffcf104` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.053479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5473b885d507428fa06011a96ffcf104)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:59.056045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:59.101805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5473b885d507428fa06011a96ffcf104', 'request_id': 'req-258e0811-4df8-4ac5-8dbd-86cd5015a936'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.110137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5473b885d507428fa06011a96ffcf104', 'request_id': 'req-258e0811-4df8-4ac5-8dbd-86cd5015a936'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.115833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5473b885d507428fa06011a96ffcf104', 'request_id': 'req-258e0811-4df8-4ac5-8dbd-86cd5015a936'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.127294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] The token cache is being invalidate because project 5473b885d507428fa06011a96ffcf104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:59.127608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.128265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.129930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1249/2503] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:59 2022] DELETE /identity/v3/projects/5473b885d507428fa06011a96ffcf104 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:59.134393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:59.143940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fce5c-755d-4370-a9ed-91643b4388bc service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:59.199559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:59.200119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.200403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.200678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] PATH_INFO: `/v3/users/a22e242ad15e44639e3ecf313b4b6537` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.201875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a22e242ad15e44639e3ecf313b4b6537)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:59.203834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:59.236379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:59.251972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a22e242ad15e44639e3ecf313b4b6537'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.259722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a22e242ad15e44639e3ecf313b4b6537'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.260310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a22e242ad15e44639e3ecf313b4b6537'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.260943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a22e242ad15e44639e3ecf313b4b6537'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.267748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56cd2cfb-bcf7-4eb0-b6f9-36c5ec0b0a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a22e242ad15e44639e3ecf313b4b6537'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.285100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1255/2504] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:12:59 2022] DELETE /identity/v3/users/a22e242ad15e44639e3ecf313b4b6537 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:59.564768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:59.566440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:59.583430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e0811-4df8-4ac5-8dbd-86cd5015a936 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:59.637937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:12:59.646185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:59.647286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.647286 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.647500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] PATH_INFO: `/v3/projects/fecd97f748e54675a9ef33aefcddee98` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.650137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fecd97f748e54675a9ef33aefcddee98)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:59.651714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:59.713851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fecd97f748e54675a9ef33aefcddee98', 'request_id': 'req-d937a605-b18d-4951-8715-e330122e13e8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.722084 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fecd97f748e54675a9ef33aefcddee98', 'request_id': 'req-d937a605-b18d-4951-8715-e330122e13e8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.729117 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fecd97f748e54675a9ef33aefcddee98', 'request_id': 'req-d937a605-b18d-4951-8715-e330122e13e8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.741325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] The token cache is being invalidate because project fecd97f748e54675a9ef33aefcddee98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:12:59.741547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.742273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-d937a605-b18d-4951-8715-e330122e13e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:12:59.743897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1250/2505] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:12:59 2022] DELETE /identity/v3/projects/fecd97f748e54675a9ef33aefcddee98 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:12:59.748840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.749432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.749432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.770023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:59.771238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:59.834688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1256/2506] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:12:59.842990 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.843283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.843561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.863605 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:12:59.864809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:12:59.872820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1251/2507] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:12:59.876919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:59.886440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56627138-d695-4a6c-98a2-4fb9caf8c5c1 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:59.890930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:59.891514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.891788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.892055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.892688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:12:59.892981 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:59.894579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:59.906228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1257/2508] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:12:59.910535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:59.920411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9983e9-d205-4668-ae9a-cf6f62d3da34 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:12:59.925066 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:12:59.925655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:12:59.925938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:12:59.926248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:12:59.926867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:12:59.928771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:12:59.955566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1252/2509] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:12:59 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:12:59.959621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:12:59.969988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa11865-30ba-4de0-80e3-7b25dca47eb8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.032279 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.032775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.033050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.033324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.033960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.035886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.057144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1258/2510] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:12:59 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:00.061249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.072120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006edc5-f83a-4848-a7b4-12b98133e605 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.076699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.077294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.077573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.077847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.078513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.080091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.090463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1253/2511] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:00.094350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.104041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91049a6e-d922-4a52-9a71-fc953fff1522 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.108594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656c082-48dc-424a-8374-583924a0cf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.109173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c082-48dc-424a-8374-583924a0cf3a admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.109446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c082-48dc-424a-8374-583924a0cf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.109715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656c082-48dc-424a-8374-583924a0cf3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.110377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656c082-48dc-424a-8374-583924a0cf3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.111935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656c082-48dc-424a-8374-583924a0cf3a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.122221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1259/2512] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:00.126315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.137024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c20f3-d0bd-4706-b7e2-aa3cd569fea1 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.141801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.142518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.142835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.143109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] PATH_INFO: `/v3/projects/86059e19d5ba46319ca8c08985e394ef/users/e27292dd0e8e4a59b012468cccc6c3a5/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.158443 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=86059e19d5ba46319ca8c08985e394ef, user_id=e27292dd0e8e4a59b012468cccc6c3a5, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.161075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.173911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1254/2513] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:00 2022] PUT /identity/v3/projects/86059e19d5ba46319ca8c08985e394ef/users/e27292dd0e8e4a59b012468cccc6c3a5/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:00.185328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.185613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.185910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.210818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:00.214635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] MFA Rules not processed for user `e27292dd0e8e4a59b012468cccc6c3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:00.217040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:00.279078 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1260/2514] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:13:00.283812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.349123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195d9bb-7b2d-495a-9e34-0178d86b55fc admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.353719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.354350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.354626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.354893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.355525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.357417 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.384378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1255/2515] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:00 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:00.388529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.398969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d46330-01a2-45c8-ad78-749e7182c28d None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.453521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.454150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.454429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.454698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.455328 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.457232 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.478665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1261/2516] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:00 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:00.482695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.493264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0b30-780d-475b-ab8c-fb9cf8db8eea admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.497841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.498476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.498756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.499026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.499656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.501230 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.502462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:00.514238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1256/2517] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:00.517766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.527457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4706e-ebc9-4550-91ab-9e839a84e24e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.532290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.532923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.533198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.533465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] PATH_INFO: `/v3/projects/8e4eb1677304462ca4b981765a5502a5/users/324a0614f87f465ebad28e6de57a618f/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.547725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4eb1677304462ca4b981765a5502a5, user_id=324a0614f87f465ebad28e6de57a618f, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.550317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.563096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1262/2518] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:00 2022] PUT /identity/v3/projects/8e4eb1677304462ca4b981765a5502a5/users/324a0614f87f465ebad28e6de57a618f/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:00.566906 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.577400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9213e-ceca-4b3c-9e8c-2d5f0ba91f70 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.631101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d2591-13d0-4c1b-9e00-aad8c1c21362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.631702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2591-13d0-4c1b-9e00-aad8c1c21362 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.631979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2591-13d0-4c1b-9e00-aad8c1c21362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.632248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2591-13d0-4c1b-9e00-aad8c1c21362 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.633458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d2591-13d0-4c1b-9e00-aad8c1c21362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.635039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d2591-13d0-4c1b-9e00-aad8c1c21362 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.644529 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1257/2519] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:13:00.648684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.659219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61125e87-5bea-4261-b459-2321eabf791e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.663775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.665082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.665375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.665640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] PATH_INFO: `/v3/projects/8e4eb1677304462ca4b981765a5502a5/users/324a0614f87f465ebad28e6de57a618f/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.667620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4eb1677304462ca4b981765a5502a5, user_id=324a0614f87f465ebad28e6de57a618f, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.670245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.680915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1263/2520] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:00 2022] PUT /identity/v3/projects/8e4eb1677304462ca4b981765a5502a5/users/324a0614f87f465ebad28e6de57a618f/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:00.691302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.691592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.691873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.718796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] MFA Rules not processed for user `324a0614f87f465ebad28e6de57a618f`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:00.720028 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:00.788377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1258/2521] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:00.800845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.802667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:00.859277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2c6da-4d1b-404d-99fe-cb55ab986552 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.862956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.863559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.863832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.864100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:00.873509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:00.874955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:00.878906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:00.886525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1264/2522] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:00.972790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:00.983008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18d14a-3960-4131-8c34-ef1fb4dc3536 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:00.986524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39ff3c-c8e2-42fe-8ed3-cfddcdb18e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:00.987125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39ff3c-c8e2-42fe-8ed3-cfddcdb18e01 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:00.987402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39ff3c-c8e2-42fe-8ed3-cfddcdb18e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:00.987674 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39ff3c-c8e2-42fe-8ed3-cfddcdb18e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:01.044278 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39ff3c-c8e2-42fe-8ed3-cfddcdb18e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:01.045706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39ff3c-c8e2-42fe-8ed3-cfddcdb18e01 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:01.054598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1259/2523] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:01.312828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:01.373389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773a2c1-3e82-408c-9ae2-701bb63c6337 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:01.377930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:01.378588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:01.378867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:01.379149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] PATH_INFO: `/v3/users/e27292dd0e8e4a59b012468cccc6c3a5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:01.380249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e27292dd0e8e4a59b012468cccc6c3a5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:01.382288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:01.427749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e27292dd0e8e4a59b012468cccc6c3a5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.435568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e27292dd0e8e4a59b012468cccc6c3a5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.436224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e27292dd0e8e4a59b012468cccc6c3a5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.436838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e27292dd0e8e4a59b012468cccc6c3a5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.443707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e27292dd0e8e4a59b012468cccc6c3a5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.460362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1265/2524] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:01 2022] DELETE /identity/v3/users/e27292dd0e8e4a59b012468cccc6c3a5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:01.465530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:01.465811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:01.466125 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:01.485337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:01.486577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:01.496012 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:01.544331 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1260/2525] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:01.553910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:01.555979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:01.612128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ed3ef-251a-4647-8b3b-057627ec558c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:01.615701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:01.616295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:01.616569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:01.616838 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:01.626514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:01.627925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:01.637112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1266/2526] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:01.896147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:01.897635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:01.911512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a75ed29-45dc-4164-8567-951857e5ec5b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:01.916656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:01.917768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:01.918092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:01.918375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] PATH_INFO: `/v3/projects/86059e19d5ba46319ca8c08985e394ef` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:01.919685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86059e19d5ba46319ca8c08985e394ef)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:01.921675 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:01.968502 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86059e19d5ba46319ca8c08985e394ef', 'request_id': 'req-61b266c9-1ef4-4330-b628-cb6009bb23d0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.975767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86059e19d5ba46319ca8c08985e394ef', 'request_id': 'req-61b266c9-1ef4-4330-b628-cb6009bb23d0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.981904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86059e19d5ba46319ca8c08985e394ef', 'request_id': 'req-61b266c9-1ef4-4330-b628-cb6009bb23d0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.992559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] The token cache is being invalidate because project 86059e19d5ba46319ca8c08985e394ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:01.992849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.993467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:01.994961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1261/2527] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:01 2022] DELETE /identity/v3/projects/86059e19d5ba46319ca8c08985e394ef => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:01.998925 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.008392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b71db-de81-4cff-b0f3-dddb26a7319e service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:02.024835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:02.065773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:02.066435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.066714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.066989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] PATH_INFO: `/v3/users/324a0614f87f465ebad28e6de57a618f` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.068094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=324a0614f87f465ebad28e6de57a618f)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:02.070040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:02.122048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324a0614f87f465ebad28e6de57a618f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.129937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324a0614f87f465ebad28e6de57a618f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.130571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324a0614f87f465ebad28e6de57a618f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.131174 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324a0614f87f465ebad28e6de57a618f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.138150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-70677fdd-50dd-4eb5-8096-3050110d2fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324a0614f87f465ebad28e6de57a618f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.155626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1267/2528] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:01 2022] DELETE /identity/v3/users/324a0614f87f465ebad28e6de57a618f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:02.430894 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.432323 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:02.445794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b266c9-1ef4-4330-b628-cb6009bb23d0 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:02.503194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:02.503816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.504090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.504358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] PATH_INFO: `/v3/projects/8e4eb1677304462ca4b981765a5502a5` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.505473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4eb1677304462ca4b981765a5502a5)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:02.507391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:02.547457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:02.554343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4eb1677304462ca4b981765a5502a5', 'request_id': 'req-db383222-4bef-49f8-8926-9652d73c65f1'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.562360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4eb1677304462ca4b981765a5502a5', 'request_id': 'req-db383222-4bef-49f8-8926-9652d73c65f1'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.567834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4eb1677304462ca4b981765a5502a5', 'request_id': 'req-db383222-4bef-49f8-8926-9652d73c65f1'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.579312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] The token cache is being invalidate because project 8e4eb1677304462ca4b981765a5502a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:02.579595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.580227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-db383222-4bef-49f8-8926-9652d73c65f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:02.581685 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1262/2529] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:02 2022] DELETE /identity/v3/projects/8e4eb1677304462ca4b981765a5502a5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:02.586565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.586851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.587125 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.606325 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:02.607534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:02.669654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1268/2530] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:02.734292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.734575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.734850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.755003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:02.756249 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:02.765069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1263/2531] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:02.770253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.780396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10aef66-85d1-4dbd-adfe-8eef2f3ed266 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:02.785313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:02.785912 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.786226 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.786500 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.787142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:13:02.787441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:02.789044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:02.800435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1269/2532] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:13:02.805336 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.814859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c423fb-e7ce-4042-bfac-3bb2527ea3a7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:02.819504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:02.820101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.820991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.821261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.821891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:02.823817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:02.850335 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1264/2533] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:02 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:02.854638 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.864477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc25339-f59f-4e27-b276-b8828a494677 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:02.889397 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:02.922950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:02.923549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.923813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.924085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.924712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:02.926634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:02.948878 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1270/2534] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:02 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:02.952637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.963060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d9afe-0edf-4067-a5ed-f3a0704bb0bc admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:02.967724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:02.968312 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:02.968586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:02.968860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:02.969489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:02.971216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:02.981897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1265/2535] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:13:02.985919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:02.996365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1992441-8a4a-47fd-9162-83de96f3f747 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:03.000937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c5ed9-554f-4dce-ac4b-9d4a35b22892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:03.001526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c5ed9-554f-4dce-ac4b-9d4a35b22892 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:03.001797 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c5ed9-554f-4dce-ac4b-9d4a35b22892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:03.002106 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c5ed9-554f-4dce-ac4b-9d4a35b22892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:03.002743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c5ed9-554f-4dce-ac4b-9d4a35b22892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:03.004334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c5ed9-554f-4dce-ac4b-9d4a35b22892 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:03.014651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1271/2536] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:03.018641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:03.028488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d0470-3211-40bf-ac28-6d6c0fbeae7a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:03.033300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:03.033943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:03.034257 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:03.034531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] PATH_INFO: `/v3/projects/ff6b0dd3fb5d490985d350f91471106a/users/e3f8df810b474deeb6ce62c37ceace14/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:03.050727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff6b0dd3fb5d490985d350f91471106a, user_id=e3f8df810b474deeb6ce62c37ceace14, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:03.053401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:03.067844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1266/2537] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:03 2022] PUT /identity/v3/projects/ff6b0dd3fb5d490985d350f91471106a/users/e3f8df810b474deeb6ce62c37ceace14/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:03.078711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:03.079052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:03.079334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:03.106829 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] MFA Rules not processed for user `e3f8df810b474deeb6ce62c37ceace14`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:03.108039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:03.170563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1272/2538] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:03.182462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:03.184137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:03.241558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805d451-2914-4673-b69b-416ed0812b33 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:03.245750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa0b3e-438b-4bfd-8b68-3def146c7910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:03.246376 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa0b3e-438b-4bfd-8b68-3def146c7910 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:03.246652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa0b3e-438b-4bfd-8b68-3def146c7910 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:03.246921 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa0b3e-438b-4bfd-8b68-3def146c7910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:03.254947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa0b3e-438b-4bfd-8b68-3def146c7910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:03.256363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa0b3e-438b-4bfd-8b68-3def146c7910 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:03.264923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1267/2539] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:13.167631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=54,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:13.261065 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:27.015430 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:27.076251 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a6962-8a02-4d1e-b28b-a137541055f9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:27.080690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:27.081308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:27.081582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:27.081851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] PATH_INFO: `/v3/users/e3f8df810b474deeb6ce62c37ceace14` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:27.083002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3f8df810b474deeb6ce62c37ceace14)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:27.084916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:27.128793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3f8df810b474deeb6ce62c37ceace14'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.136961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3f8df810b474deeb6ce62c37ceace14'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.137578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3f8df810b474deeb6ce62c37ceace14'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.138212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3f8df810b474deeb6ce62c37ceace14'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.145402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3f8df810b474deeb6ce62c37ceace14'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.162620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1273/2540] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:27 2022] DELETE /identity/v3/users/e3f8df810b474deeb6ce62c37ceace14 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:27.167593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:27.168075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:27.168177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:27.187986 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:27.189214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:27.245782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1268/2541] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:27.255594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:27.257029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:27.313676 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6fc84-c518-48ae-87c8-598f0639eb4b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:27.317385 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1759c94f-da35-4a51-a661-3c5982ebb78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:27.317984 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c94f-da35-4a51-a661-3c5982ebb78f service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:27.318294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c94f-da35-4a51-a661-3c5982ebb78f service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:27.318566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1759c94f-da35-4a51-a661-3c5982ebb78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:27.327541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c94f-da35-4a51-a661-3c5982ebb78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:27.328951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1759c94f-da35-4a51-a661-3c5982ebb78f service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:27.338684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1274/2542] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:27.846139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:27.847626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:27.860204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a68d9-a6af-4894-b0a7-0bbee757e53e None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:27.864813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:27.865446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:27.865722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:27.866014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] PATH_INFO: `/v3/projects/ff6b0dd3fb5d490985d350f91471106a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:27.867145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff6b0dd3fb5d490985d350f91471106a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:27.869149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:27.916086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff6b0dd3fb5d490985d350f91471106a', 'request_id': 'req-4cae34f9-26a8-44b6-b870-9ecb3c12c494'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.923593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff6b0dd3fb5d490985d350f91471106a', 'request_id': 'req-4cae34f9-26a8-44b6-b870-9ecb3c12c494'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.929166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff6b0dd3fb5d490985d350f91471106a', 'request_id': 'req-4cae34f9-26a8-44b6-b870-9ecb3c12c494'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.939857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] The token cache is being invalidate because project ff6b0dd3fb5d490985d350f91471106a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:27.940147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.940767 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-4cae34f9-26a8-44b6-b870-9ecb3c12c494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:27.942253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1269/2543] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:27 2022] DELETE /identity/v3/projects/ff6b0dd3fb5d490985d350f91471106a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:27.947690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:27.947979 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:27.948258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:27.969332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:27.970582 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:27.987004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:28.029887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1275/2544] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:28.037878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.038209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.038486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.058179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:28.059423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:28.067179 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1270/2545] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:28.071888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.081349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9b42c-fadf-4de7-8571-482aeecbf70a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.085859 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.086477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.086750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.087020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.087659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:13:28.087955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.089542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.100632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1276/2546] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:13:28.104737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.113717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3e5c-910f-4567-b306-c7b6129af938 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.118886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.119473 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.119751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.120021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.120652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.122566 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.150010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1271/2547] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:28 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:28.155723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.165746 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2392a2ce-3cd0-44ca-96bb-8d0067875fc9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.220758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.221366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.221643 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.221914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.222586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.224504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.246379 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1277/2548] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:28 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:28.250452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.261360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67f58-9c73-4b70-87b2-24a95067a07d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.266056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.266684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.266961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.267234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.267870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.269488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.279274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1272/2549] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:28.283225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.292742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96234a61-d51a-428f-a895-cf96a4b5f6a0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.297305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe38a8-d0c4-4a2e-8775-2d4cf2327dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.297919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe38a8-d0c4-4a2e-8775-2d4cf2327dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.298240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe38a8-d0c4-4a2e-8775-2d4cf2327dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.298508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe38a8-d0c4-4a2e-8775-2d4cf2327dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.299136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe38a8-d0c4-4a2e-8775-2d4cf2327dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.300745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe38a8-d0c4-4a2e-8775-2d4cf2327dd8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.311961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1278/2550] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:28.315866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.326055 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47b79-5b5f-4594-a5ff-f95575bc75a3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.330655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.331293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.331571 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.331843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] PATH_INFO: `/v3/projects/33ebabdd38da434493e48995bcd4120e/users/1655f031016b4cbeb6a15549fc45258f/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.346867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ebabdd38da434493e48995bcd4120e, user_id=1655f031016b4cbeb6a15549fc45258f, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.349495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.362458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1273/2551] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:28 2022] PUT /identity/v3/projects/33ebabdd38da434493e48995bcd4120e/users/1655f031016b4cbeb6a15549fc45258f/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:28.373115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.373403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.373674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.400236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] MFA Rules not processed for user `1655f031016b4cbeb6a15549fc45258f`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:28.401669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:28.463815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1279/2552] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:28.476411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.478962 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:28.501317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:28.537606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0399ebf-9a2f-4c7e-84eb-0c4cb133b86e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.541706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63af09a-c593-476c-a998-ff3cb16f469f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.542346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63af09a-c593-476c-a998-ff3cb16f469f service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.542626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63af09a-c593-476c-a998-ff3cb16f469f service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.542898 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63af09a-c593-476c-a998-ff3cb16f469f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.551074 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63af09a-c593-476c-a998-ff3cb16f469f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.552498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63af09a-c593-476c-a998-ff3cb16f469f service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.561903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1274/2553] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:28.586260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.605184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:28.650628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03028d-d65f-4647-b368-e9e2ae30365b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.654121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.654748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.655019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.655294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] PATH_INFO: `/v3/users/1655f031016b4cbeb6a15549fc45258f` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.656674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1655f031016b4cbeb6a15549fc45258f)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.658657 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.705038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1655f031016b4cbeb6a15549fc45258f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:28.712291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1655f031016b4cbeb6a15549fc45258f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:28.712914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1655f031016b4cbeb6a15549fc45258f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:28.713542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1655f031016b4cbeb6a15549fc45258f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:28.720598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1655f031016b4cbeb6a15549fc45258f'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:28.738475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1280/2554] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:28 2022] DELETE /identity/v3/users/1655f031016b4cbeb6a15549fc45258f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:28.743482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.743765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.744037 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.762485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:28.763676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:28.829956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1275/2555] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:28.841747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:28.843424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:28.900857 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948361cd-a44e-4bd2-9b0e-8f5bd1272181 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:28.904433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7241d9-191a-485d-af67-161754451639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:28.905026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7241d9-191a-485d-af67-161754451639 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:28.905301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7241d9-191a-485d-af67-161754451639 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:28.905569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7241d9-191a-485d-af67-161754451639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:28.915223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7241d9-191a-485d-af67-161754451639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:28.916635 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7241d9-191a-485d-af67-161754451639 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:28.925711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1281/2556] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:29.187742 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.189238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:29.202277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883a5db-0760-4e7e-b14d-2b1fa2577a18 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.207027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.207830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.208173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.208513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] PATH_INFO: `/v3/projects/33ebabdd38da434493e48995bcd4120e` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.209709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ebabdd38da434493e48995bcd4120e)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.212328 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.259512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ebabdd38da434493e48995bcd4120e', 'request_id': 'req-643d3c2a-ada6-4d71-b463-2f75630f7e88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:29.266721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ebabdd38da434493e48995bcd4120e', 'request_id': 'req-643d3c2a-ada6-4d71-b463-2f75630f7e88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:29.272354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ebabdd38da434493e48995bcd4120e', 'request_id': 'req-643d3c2a-ada6-4d71-b463-2f75630f7e88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:29.284013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] The token cache is being invalidate because project 33ebabdd38da434493e48995bcd4120e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:29.284300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:29.284965 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-643d3c2a-ada6-4d71-b463-2f75630f7e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:29.286475 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1276/2557] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:29 2022] DELETE /identity/v3/projects/33ebabdd38da434493e48995bcd4120e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:29.291685 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.292157 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.292261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.311165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:29.312346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:29.345173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:29.371551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1282/2558] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:29.379226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.379514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.379791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.400114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:29.401322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:29.409439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1277/2559] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:29.413437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.422784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2bc9a-2366-4ef4-b7b9-6de523ec39af None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.427572 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.428165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.428444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.428717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.429362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:13:29.429662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.431296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.442548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1283/2560] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:13:29.447140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.456177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1330a5-5935-4163-85af-b64fbdfd9eab None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.460822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.462040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.462364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.462636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.463277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.465354 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.491783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1278/2561] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:29 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:29.496634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.506495 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bdcda-e6d9-4499-b273-20a872439ef6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.562176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.562772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.563047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.563315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.563939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.565848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.586760 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1284/2562] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:29 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:29.590902 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.601184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d9d62-0f15-4652-991e-f9961b3088c4 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.605841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.606470 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.606754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.607025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.607657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.609251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.620133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1279/2563] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:29.623898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.634208 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa57d1-e7e7-468e-9f8e-c07823ff56f9 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.638728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445a696-09a9-4956-a9fb-0384bbfeeffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.639312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a696-09a9-4956-a9fb-0384bbfeeffe admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.639590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a696-09a9-4956-a9fb-0384bbfeeffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.639862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445a696-09a9-4956-a9fb-0384bbfeeffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.640493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445a696-09a9-4956-a9fb-0384bbfeeffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.642092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445a696-09a9-4956-a9fb-0384bbfeeffe admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.653631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1285/2564] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:29.658090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.668041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b451fe5-1d56-464a-a263-37eb58878eef admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.672601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.673248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.673530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.673808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] PATH_INFO: `/v3/projects/2e82c8cb690549b99a512f13d953fe1a/users/9669a833fc854fd69f54d1726ed7d5c2/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.676023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:29.691148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e82c8cb690549b99a512f13d953fe1a, user_id=9669a833fc854fd69f54d1726ed7d5c2, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.693851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.706559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1280/2565] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:29 2022] PUT /identity/v3/projects/2e82c8cb690549b99a512f13d953fe1a/users/9669a833fc854fd69f54d1726ed7d5c2/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:29.717840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.718156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.718433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.745869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] MFA Rules not processed for user `9669a833fc854fd69f54d1726ed7d5c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:29.747107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:29.809088 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1286/2566] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:29.819664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:29.821952 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:29.877973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52763e69-604c-4c09-be01-f375134d907f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:29.881497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b65b0-8125-47fa-9a4a-b7b84e8ed4d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:29.882124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b65b0-8125-47fa-9a4a-b7b84e8ed4d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:29.882401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b65b0-8125-47fa-9a4a-b7b84e8ed4d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:29.882669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b65b0-8125-47fa-9a4a-b7b84e8ed4d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:29.890694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b65b0-8125-47fa-9a4a-b7b84e8ed4d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:29.892104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b65b0-8125-47fa-9a4a-b7b84e8ed4d4 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:29.900710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1281/2567] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:39.748155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:39.786216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:39.810156 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dcca90-8ba8-4f0d-b509-603341e006ff None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:39.814752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:39.815375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:39.815660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:39.815930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] PATH_INFO: `/v3/users/9669a833fc854fd69f54d1726ed7d5c2` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:39.817045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9669a833fc854fd69f54d1726ed7d5c2)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:39.819004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:39.864380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9669a833fc854fd69f54d1726ed7d5c2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:39.872129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9669a833fc854fd69f54d1726ed7d5c2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:39.872781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9669a833fc854fd69f54d1726ed7d5c2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:39.873392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9669a833fc854fd69f54d1726ed7d5c2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:39.880151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9669a833fc854fd69f54d1726ed7d5c2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:39.896830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:39.905669 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1287/2568] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:39 2022] DELETE /identity/v3/users/9669a833fc854fd69f54d1726ed7d5c2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:39.910460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:39.910748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:39.911024 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:39.930896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:39.932092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:39.988964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1282/2569] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:39.999585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:40.000973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:40.056849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a73832-6969-485d-9929-e7748c91ba56 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:40.060424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:40.061014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:40.061287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:40.061583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:40.070342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:40.071780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:40.080957 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1288/2570] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:40.345154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:40.346733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:40.359333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf4922-1de9-461e-8824-339b60dcdadc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:40.365067 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:40.365701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:40.365983 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:40.366271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] PATH_INFO: `/v3/projects/2e82c8cb690549b99a512f13d953fe1a` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:40.367401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e82c8cb690549b99a512f13d953fe1a)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:40.370113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:40.417783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e82c8cb690549b99a512f13d953fe1a', 'request_id': 'req-313c138e-a574-48a0-a948-99abc5bdb021'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:40.424900 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e82c8cb690549b99a512f13d953fe1a', 'request_id': 'req-313c138e-a574-48a0-a948-99abc5bdb021'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:40.430575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e82c8cb690549b99a512f13d953fe1a', 'request_id': 'req-313c138e-a574-48a0-a948-99abc5bdb021'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:40.441504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] The token cache is being invalidate because project 2e82c8cb690549b99a512f13d953fe1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:40.441793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:40.442439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:40.443955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1283/2571] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:40 2022] DELETE /identity/v3/projects/2e82c8cb690549b99a512f13d953fe1a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:48.229960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:48.239048 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e29e3d-b923-4b04-9a6a-dc5a2531b0d3 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:48.287216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:48.287814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:48.288090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:48.288360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:48.290117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:48.299913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:48.352729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:48.354193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:48.363873 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1289/2572] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:50.438459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:51.466622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:51.468771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:51.486992 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c138e-a574-48a0-a948-99abc5bdb021 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:51.491717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:51.492528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:51.493013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:51.493352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] PATH_INFO: `/v3/users/0948754217fc4ff2ba0241fa35feaf90` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:51.495115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0948754217fc4ff2ba0241fa35feaf90)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:51.497130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:51.557525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0948754217fc4ff2ba0241fa35feaf90'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.565979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0948754217fc4ff2ba0241fa35feaf90'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.566288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0948754217fc4ff2ba0241fa35feaf90'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.573325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0948754217fc4ff2ba0241fa35feaf90'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.573957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1d5f756f-a6e1-4d4b-96ac-651792b96ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0948754217fc4ff2ba0241fa35feaf90'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.590897 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1284/2573] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:51 2022] DELETE /identity/v3/users/0948754217fc4ff2ba0241fa35feaf90 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:51.761433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:51.773658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230d198-d53f-4ffa-8cd8-cdd8e6029c56 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:51.833487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:51.834155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:51.834435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:51.834702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] PATH_INFO: `/v3/projects/c1aaf20f480a423ea5feca81585aeeed` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:51.835815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1aaf20f480a423ea5feca81585aeeed)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:51.837830 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:51.901540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1aaf20f480a423ea5feca81585aeeed', 'request_id': 'req-09f17364-027c-4b84-ba02-48a005015f7a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.907856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1aaf20f480a423ea5feca81585aeeed', 'request_id': 'req-09f17364-027c-4b84-ba02-48a005015f7a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.915087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1aaf20f480a423ea5feca81585aeeed', 'request_id': 'req-09f17364-027c-4b84-ba02-48a005015f7a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.927038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] The token cache is being invalidate because project c1aaf20f480a423ea5feca81585aeeed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:51.927351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.927977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-09f17364-027c-4b84-ba02-48a005015f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:51.929443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1290/2574] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:51 2022] DELETE /identity/v3/projects/c1aaf20f480a423ea5feca81585aeeed => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:51.934942 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:51.935225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:51.935497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:51.954436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:51.955620 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.013660 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1285/2575] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:52.022624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.022910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.023195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.043764 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:52.044961 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.052778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1291/2576] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:52.056957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.066642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2304f748-42af-49c5-988d-2cc8c0f21c73 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.071129 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.071743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.071999 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.072270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.072910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:13:52.073211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.074842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.086101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1286/2577] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:13:52.090824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.099959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44fe71-3caa-4add-bc6d-b48709d22b38 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.105219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.105806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.106119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.106392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.107012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.108890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.134672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1292/2578] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:52.138796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.148557 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c4f6e9-7fd6-4e6b-b570-288f0d3d2617 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.203640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.204253 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.204549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.204800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.205446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.207395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.215783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:52.230874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1287/2579] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:52.234956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.245372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae0cad-2cc1-4b7e-bf7d-b3e89bd4457c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.250016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.250614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.250890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.251160 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.251785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.253347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.255352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:52.264438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1293/2580] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:52.268004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.278293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5334a07a-4e1f-4a76-b90b-2d1990b7f1a7 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.283141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70459aa0-50fd-4fb7-a8ab-a781f0b2d1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.283730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70459aa0-50fd-4fb7-a8ab-a781f0b2d1de admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.284011 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70459aa0-50fd-4fb7-a8ab-a781f0b2d1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.284283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70459aa0-50fd-4fb7-a8ab-a781f0b2d1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.284914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70459aa0-50fd-4fb7-a8ab-a781f0b2d1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.286538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70459aa0-50fd-4fb7-a8ab-a781f0b2d1de admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.297497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1288/2581] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:52.301567 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.311289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77020ef-2fe1-4a76-9060-af830cbfbeab admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.315809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.316448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.316723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.316989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] PATH_INFO: `/v3/projects/6ffd6761edd64ac8a3dcab4c557eb15c/users/408167f330884552990d2e0fca2673b8/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.332642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ffd6761edd64ac8a3dcab4c557eb15c, user_id=408167f330884552990d2e0fca2673b8, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.335277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.348673 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1294/2582] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:52 2022] PUT /identity/v3/projects/6ffd6761edd64ac8a3dcab4c557eb15c/users/408167f330884552990d2e0fca2673b8/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:52.358714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.358987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.359261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.387208 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] MFA Rules not processed for user `408167f330884552990d2e0fca2673b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:52.388404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.451592 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1289/2583] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:52.456043 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.517417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5ec39-d31f-45e8-9dea-d9033ffd805b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.521958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.522595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.522869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.523137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.523756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.526276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.551973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1295/2584] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:52.556031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.568106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bba9a-bb28-4344-be3a-073fc70905c6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.624014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.624609 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.624884 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.625151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.625777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.627715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.648166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1290/2585] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:52.653441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.663550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec3297-57ec-44ab-8558-4f5aa29d8876 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.668811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.669488 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.669758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.670115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.670861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.672501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.683547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1296/2586] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:52.687778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.698358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35cdab-ced2-438b-8f1b-88c7d4227b6c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.702953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.703599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.703881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.704158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] PATH_INFO: `/v3/projects/acea8ddcce3b41808bc32509f18948f5/users/d76a202eff6b4d19a0852e5820d45e2b/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.718744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acea8ddcce3b41808bc32509f18948f5, user_id=d76a202eff6b4d19a0852e5820d45e2b, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.721449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.735359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1291/2587] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:52 2022] PUT /identity/v3/projects/acea8ddcce3b41808bc32509f18948f5/users/d76a202eff6b4d19a0852e5820d45e2b/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:52.739192 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.749575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b82801-d313-4950-a060-9c6cb340e802 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.807475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60056aa5-af76-4d62-8256-b921c3e10d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.808103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60056aa5-af76-4d62-8256-b921c3e10d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.808454 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60056aa5-af76-4d62-8256-b921c3e10d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.808644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60056aa5-af76-4d62-8256-b921c3e10d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.809388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60056aa5-af76-4d62-8256-b921c3e10d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.811017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60056aa5-af76-4d62-8256-b921c3e10d4b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.823282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1297/2588] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:52.827449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.839837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6df5b7-712f-445b-b1b5-315fd1179c14 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.845689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.846538 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.846846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.847118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] PATH_INFO: `/v3/projects/acea8ddcce3b41808bc32509f18948f5/users/d76a202eff6b4d19a0852e5820d45e2b/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.849016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acea8ddcce3b41808bc32509f18948f5, user_id=d76a202eff6b4d19a0852e5820d45e2b, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.851696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.864126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1292/2589] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:52 2022] PUT /identity/v3/projects/acea8ddcce3b41808bc32509f18948f5/users/d76a202eff6b4d19a0852e5820d45e2b/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:52.875407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.875698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.875974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.883829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.885573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.906525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] MFA Rules not processed for user `d76a202eff6b4d19a0852e5820d45e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:52.907759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.909265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:52.940705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:52.960215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6f5dc-bbc5-4f5e-b9cb-e5fbd6c81376 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:52.965856 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:52.966499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:52.966774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:52.967045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:52.968779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.977485 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1298/2590] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:52.981118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:52.982862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:52.989103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:52.990459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:52.993193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1293/2591] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:13:53.049512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3eb819-1c45-4878-90f8-d31544a78bc7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:53.053435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b915d-dde3-444c-b439-dd3db2f17456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:53.054067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b915d-dde3-444c-b439-dd3db2f17456 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:53.054343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b915d-dde3-444c-b439-dd3db2f17456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:53.054612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b915d-dde3-444c-b439-dd3db2f17456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:53.064027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b915d-dde3-444c-b439-dd3db2f17456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:53.065428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b915d-dde3-444c-b439-dd3db2f17456 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:53.075326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1299/2592] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:13:53.461938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:53.523704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ebbf5-e750-462a-9273-572845c44aa7 service placement] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:53.528642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:53.529259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:53.529535 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:53.529806 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] PATH_INFO: `/v3/users/408167f330884552990d2e0fca2673b8` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:53.531186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=408167f330884552990d2e0fca2673b8)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:53.533086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:53.583547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '408167f330884552990d2e0fca2673b8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:53.591090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '408167f330884552990d2e0fca2673b8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:53.591765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '408167f330884552990d2e0fca2673b8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:53.598645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '408167f330884552990d2e0fca2673b8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:53.599262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '408167f330884552990d2e0fca2673b8'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:53.616339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1294/2593] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:53 2022] DELETE /identity/v3/users/408167f330884552990d2e0fca2673b8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:53.620714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:53.620993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:53.621260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:53.639594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:53.640774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:53.699810 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1300/2594] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:53.709373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:53.710829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:53.742799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:53.769186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccdd2a2-eb37-4c7f-b24f-d8b95b86fa55 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:53.772872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:53.773474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:53.773753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:53.774051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:53.783718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:53.785141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:53.795161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1295/2595] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:54.057411 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:54.058870 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:54.073868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ba2c7-1ed9-4b06-99e5-405660145b40 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:54.081242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:54.082437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.082550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.082809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] PATH_INFO: `/v3/projects/6ffd6761edd64ac8a3dcab4c557eb15c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.083973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ffd6761edd64ac8a3dcab4c557eb15c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:54.086417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:54.132863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ffd6761edd64ac8a3dcab4c557eb15c', 'request_id': 'req-6d3bc915-a834-463a-8ca1-97785f6719d0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.138805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ffd6761edd64ac8a3dcab4c557eb15c', 'request_id': 'req-6d3bc915-a834-463a-8ca1-97785f6719d0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.145836 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ffd6761edd64ac8a3dcab4c557eb15c', 'request_id': 'req-6d3bc915-a834-463a-8ca1-97785f6719d0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.147679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:54.158982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] The token cache is being invalidate because project 6ffd6761edd64ac8a3dcab4c557eb15c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:54.159269 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.159885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.161577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1301/2596] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:54 2022] DELETE /identity/v3/projects/6ffd6761edd64ac8a3dcab4c557eb15c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:54.165410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:54.174804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c8660-3784-4780-b40b-92dd2af39c1b service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:54.229005 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:54.229628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.229905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.230205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] PATH_INFO: `/v3/users/d76a202eff6b4d19a0852e5820d45e2b` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.231303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76a202eff6b4d19a0852e5820d45e2b)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:54.233224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:54.278224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76a202eff6b4d19a0852e5820d45e2b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.286227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76a202eff6b4d19a0852e5820d45e2b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.286844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76a202eff6b4d19a0852e5820d45e2b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.293692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76a202eff6b4d19a0852e5820d45e2b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.294351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-618b6405-92e0-4bb9-be67-10656881521e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76a202eff6b4d19a0852e5820d45e2b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.315617 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1296/2597] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:54 2022] DELETE /identity/v3/users/d76a202eff6b4d19a0852e5820d45e2b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:54.606429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:54.607885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:54.624553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3bc915-a834-463a-8ca1-97785f6719d0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:54.691782 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:54.692423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.692698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.693121 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] PATH_INFO: `/v3/projects/acea8ddcce3b41808bc32509f18948f5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.694314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] RBAC: Authorizing `identity:delete_project(project_id=acea8ddcce3b41808bc32509f18948f5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:54.696308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:54.742684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acea8ddcce3b41808bc32509f18948f5', 'request_id': 'req-5155430a-625c-4edf-a874-cd4af2db553c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.748159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acea8ddcce3b41808bc32509f18948f5', 'request_id': 'req-5155430a-625c-4edf-a874-cd4af2db553c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.755082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acea8ddcce3b41808bc32509f18948f5', 'request_id': 'req-5155430a-625c-4edf-a874-cd4af2db553c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.767072 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] The token cache is being invalidate because project acea8ddcce3b41808bc32509f18948f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:54.767355 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.767973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-5155430a-625c-4edf-a874-cd4af2db553c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:54.769432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1302/2598] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:54 2022] DELETE /identity/v3/projects/acea8ddcce3b41808bc32509f18948f5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:54.774431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.774882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.774977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.794797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:54.795976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:54.841063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:54.857098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1297/2599] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:54.864840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66145e91-401b-4085-bc93-113282f9c730 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.865123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66145e91-401b-4085-bc93-113282f9c730 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.865396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66145e91-401b-4085-bc93-113282f9c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.886352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-66145e91-401b-4085-bc93-113282f9c730 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:54.887597 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-66145e91-401b-4085-bc93-113282f9c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:54.895378 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1303/2600] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:54.899471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:54.910026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb1cc6-f57e-4dea-b2f2-d0189551a8bb None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:54.914853 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:54.915466 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.915746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.916022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.916668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:13:54.916971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:54.918612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:54.930140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1298/2601] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:13:54.934744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66145e91-401b-4085-bc93-113282f9c730 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:54.944179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66145e91-401b-4085-bc93-113282f9c730 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:54.949511 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:54.950143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:54.950424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:54.950696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:54.952521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:54.953204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:54.979220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1304/2602] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:54 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:54.983433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:54.993183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03ba64-01c5-4dfa-8d1e-9cc06fa09471 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.051970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.052585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.052872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.053147 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.053775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.055743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.077430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1299/2603] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:54 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:55.081620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.091777 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc336a32-b1ea-4d12-85a9-75309f980995 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.096352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.096945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.097234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.097501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.098170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.099784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.110017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1305/2604] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:55.113845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.124367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bf67f-bee0-4b47-9d13-9d50bda2f8e7 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.129217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e084d-af51-4994-89a3-effc707b2361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.129820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e084d-af51-4994-89a3-effc707b2361 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.130126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e084d-af51-4994-89a3-effc707b2361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.130399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e084d-af51-4994-89a3-effc707b2361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.131026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e084d-af51-4994-89a3-effc707b2361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.132614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e084d-af51-4994-89a3-effc707b2361 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.143946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1300/2605] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:55.148330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.159294 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c71f7-fdb5-47be-83f5-1c76a1d951fe admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.164061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.164708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.165431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.165431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] PATH_INFO: `/v3/projects/2cbd56cd2a2e48f0926073d3310f2402/users/5b49bc4eeb404f208e8dc0e9c3874073/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.179540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cbd56cd2a2e48f0926073d3310f2402, user_id=5b49bc4eeb404f208e8dc0e9c3874073, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.182721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.195265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1306/2606] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:55 2022] PUT /identity/v3/projects/2cbd56cd2a2e48f0926073d3310f2402/users/5b49bc4eeb404f208e8dc0e9c3874073/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:55.206126 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.206419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.206703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.233645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] MFA Rules not processed for user `5b49bc4eeb404f208e8dc0e9c3874073`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:55.234893 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:55.302056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1301/2607] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:55.307483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.332617 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:55.371536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b1421c-1adf-42e6-8a65-818cfafb9ec6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.376165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.376793 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.377073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.377346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.377972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.379939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.405366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1307/2608] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:55 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:55.409759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.420974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65616153-22af-47ac-bb0f-cd5d4c3ea614 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.463349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:55.477650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.478287 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.479106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.479705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.480261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.482237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.503165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1302/2609] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:55 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:55.507314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.517545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd581f-f3ad-479e-945b-ecd7b2230dd0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.522159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.522748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.523027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.523297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.523927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.525510 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.536285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1308/2610] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:55.540302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.550570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd8264-e350-4a82-86ea-910f5cc0eec9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.555113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.555751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.556025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.556293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] PATH_INFO: `/v3/projects/78d3e977503a47fe842e646d0436fb15/users/68a1b0912f7b463891d0aff4617cffcc/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.571045 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d3e977503a47fe842e646d0436fb15, user_id=68a1b0912f7b463891d0aff4617cffcc, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.573707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.585719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1303/2611] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:55 2022] PUT /identity/v3/projects/78d3e977503a47fe842e646d0436fb15/users/68a1b0912f7b463891d0aff4617cffcc/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:55.589499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.599275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778eda37-6c8a-4ab6-82e6-bc55c21e5195 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.659273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064da2d-1d02-42bd-9494-08e32a714d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.660118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064da2d-1d02-42bd-9494-08e32a714d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.660611 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064da2d-1d02-42bd-9494-08e32a714d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.660958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064da2d-1d02-42bd-9494-08e32a714d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.661671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064da2d-1d02-42bd-9494-08e32a714d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.663371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064da2d-1d02-42bd-9494-08e32a714d98 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.674071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1309/2612] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:55.677933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.688123 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6beb539-bab6-4ed4-8e50-b92c6547334d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.692732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.693378 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.693655 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.693924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] PATH_INFO: `/v3/projects/78d3e977503a47fe842e646d0436fb15/users/68a1b0912f7b463891d0aff4617cffcc/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.695981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d3e977503a47fe842e646d0436fb15, user_id=68a1b0912f7b463891d0aff4617cffcc, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.698678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.708872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1304/2613] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:55 2022] PUT /identity/v3/projects/78d3e977503a47fe842e646d0436fb15/users/68a1b0912f7b463891d0aff4617cffcc/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:55.720667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.720968 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.721238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.750910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] MFA Rules not processed for user `68a1b0912f7b463891d0aff4617cffcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:55.752136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:55.770879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.772709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:55.828551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1310/2614] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:55.841030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:55.842577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:55.844231 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e9ad6-0b56-472d-b0c8-2dd3bd2b20d8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.848261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.848904 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.849178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.849503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.852174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:55.855220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5217cdb-d3b7-46b3-b2e5-bba5c5bfc402 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:55.858897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:55.859498 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:55.859809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:55.860096 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:55.870149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.872193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.882041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1311/2615] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:55.918533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:55.918533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3186384-0d8f-42c0-8ec6-3d1f8ea368f0 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:55.929340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1305/2616] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:13:56.424173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:56.429465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:56.487698 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031181c-a1d8-4e00-9bb3-3c5602af67a8 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:56.492050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:56.492697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:56.492973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:56.493238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] PATH_INFO: `/v3/users/5b49bc4eeb404f208e8dc0e9c3874073` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:56.494863 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b49bc4eeb404f208e8dc0e9c3874073)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:56.496756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:56.546751 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b49bc4eeb404f208e8dc0e9c3874073'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:56.555143 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b49bc4eeb404f208e8dc0e9c3874073'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:56.555861 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b49bc4eeb404f208e8dc0e9c3874073'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:56.556496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b49bc4eeb404f208e8dc0e9c3874073'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:56.565497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b49bc4eeb404f208e8dc0e9c3874073'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:56.584133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1312/2617] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:56 2022] DELETE /identity/v3/users/5b49bc4eeb404f208e8dc0e9c3874073 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:56.588648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:56.588932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:56.589205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:56.609949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:56.611238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:56.667533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:56.685659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1306/2618] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:56.693697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:56.695189 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:56.756424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0dc684-0a50-4779-9b54-58b07c618589 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:56.760791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:56.761400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:56.761702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:56.761977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:56.772972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:56.774464 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:56.784683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1313/2619] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:57.055928 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:57.058156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:57.073572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a68c3a-24e5-424a-b3e2-3019aaaa15cc None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:57.079195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:57.079943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.080227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.080494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] PATH_INFO: `/v3/projects/2cbd56cd2a2e48f0926073d3310f2402` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.081615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cbd56cd2a2e48f0926073d3310f2402)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:57.083610 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:57.132307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cbd56cd2a2e48f0926073d3310f2402', 'request_id': 'req-b73b139c-155a-46bf-b366-a22e6f5a1d1f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.140320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cbd56cd2a2e48f0926073d3310f2402', 'request_id': 'req-b73b139c-155a-46bf-b366-a22e6f5a1d1f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.146367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cbd56cd2a2e48f0926073d3310f2402', 'request_id': 'req-b73b139c-155a-46bf-b366-a22e6f5a1d1f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.159386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] The token cache is being invalidate because project 2cbd56cd2a2e48f0926073d3310f2402 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:57.159677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.160315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.161807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1307/2620] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:57 2022] DELETE /identity/v3/projects/2cbd56cd2a2e48f0926073d3310f2402 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:57.176268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:57.186602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bda890-d3a0-484c-a0ed-71d3ef13d3fe service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:57.209261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:57.244212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:57.244853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.245133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.245408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] PATH_INFO: `/v3/users/68a1b0912f7b463891d0aff4617cffcc` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.246561 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a1b0912f7b463891d0aff4617cffcc)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:57.248517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:57.294947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a1b0912f7b463891d0aff4617cffcc'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.301737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a1b0912f7b463891d0aff4617cffcc'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.302372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a1b0912f7b463891d0aff4617cffcc'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.302970 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a1b0912f7b463891d0aff4617cffcc'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.309688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d2c93292-ee9f-4ebc-bc9f-83ed721a4f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a1b0912f7b463891d0aff4617cffcc'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.326246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1314/2621] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:57 2022] DELETE /identity/v3/users/68a1b0912f7b463891d0aff4617cffcc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:57.590166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:57.591582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:57.606581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73b139c-155a-46bf-b366-a22e6f5a1d1f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:57.666761 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:57.667399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.667673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.667941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] PATH_INFO: `/v3/projects/78d3e977503a47fe842e646d0436fb15` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.669049 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d3e977503a47fe842e646d0436fb15)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:57.670981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:57.716258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d3e977503a47fe842e646d0436fb15', 'request_id': 'req-5f00336d-43d3-4abf-89d4-ee9957d8db80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.723673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d3e977503a47fe842e646d0436fb15', 'request_id': 'req-5f00336d-43d3-4abf-89d4-ee9957d8db80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.730013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d3e977503a47fe842e646d0436fb15', 'request_id': 'req-5f00336d-43d3-4abf-89d4-ee9957d8db80'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.741116 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] The token cache is being invalidate because project 78d3e977503a47fe842e646d0436fb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:57.741387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.742025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5f00336d-43d3-4abf-89d4-ee9957d8db80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:57.743479 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1308/2622] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:57 2022] DELETE /identity/v3/projects/78d3e977503a47fe842e646d0436fb15 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:57.748891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.749350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.749460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.769499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:57.770714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:57.827715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1315/2623] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:57.835412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.835693 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.835964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.850194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:57.856699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:57.858348 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:57.866601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1309/2624] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:57.871457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:57.881002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a32636-32a4-4811-84b1-ae8a35574e46 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:57.885540 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:57.886171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.886449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.886722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.887365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:13:57.887665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:57.889235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:57.900956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1316/2625] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:13:57.905226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:57.917971 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c9c28-e444-4df2-b515-97c3417f10f2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:57.922776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:57.923383 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:57.923663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:57.923936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:57.924582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:57.926576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:57.954060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1310/2626] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:57 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:13:57.958212 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:57.968605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c777ee11-b324-4840-bd85-fc68e2276086 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:57.998529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:58.027915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.028527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.028805 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.029062 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.029699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.031886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.052010 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1317/2627] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:57 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:58.056169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.066385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90326f9c-cde5-4c73-bf39-3f21b883892e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.071299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.071896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.072205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.072500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.073196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.074802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.084696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1311/2628] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:58.088894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.098653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ea85d-4f56-4a9f-815f-03433743b639 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.103193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f0432-b71a-4963-a55d-3e38cf952f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.104427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f0432-b71a-4963-a55d-3e38cf952f17 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.104699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f0432-b71a-4963-a55d-3e38cf952f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.104967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f0432-b71a-4963-a55d-3e38cf952f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.105593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f0432-b71a-4963-a55d-3e38cf952f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.107267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f0432-b71a-4963-a55d-3e38cf952f17 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.116788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1318/2629] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:58.120649 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.130349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214907d-0569-498a-ba06-1ed6b4bbe2e4 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.134948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.135584 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.135860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.136240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] PATH_INFO: `/v3/projects/a79c2381a7c8414a8f91150076e51a22/users/5dd8d2e3e02b48e499dcaf78be1862df/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.151073 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79c2381a7c8414a8f91150076e51a22, user_id=5dd8d2e3e02b48e499dcaf78be1862df, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.153697 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.166630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1312/2630] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:58 2022] PUT /identity/v3/projects/a79c2381a7c8414a8f91150076e51a22/users/5dd8d2e3e02b48e499dcaf78be1862df/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:58.176928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.177214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.177487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.207574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] MFA Rules not processed for user `5dd8d2e3e02b48e499dcaf78be1862df`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:58.208823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:58.278261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1319/2631] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:58.283093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.344302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a14190-4f27-48c1-ac33-476b12cecf31 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.348917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.349524 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.349805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.350107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.350745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.352647 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.377697 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1313/2632] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:13:58 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:58.381820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.392875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681987a-0455-4f0e-a132-b6463ac9653c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.447947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.448514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.448788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.449056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.449678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.451630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.472364 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1320/2633] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:13:58 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:13:58.475927 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.485937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed9058-50cd-46c4-acc1-9049dbec710e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.490437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.491029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.491303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.491570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.492197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.493768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.504999 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1314/2634] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:58.508886 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.519058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6c07-6125-480c-a2ea-0cc38edc6708 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.523590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.524223 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.524499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.524773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] PATH_INFO: `/v3/projects/8c14b1c27d4f43508818f2d87d4b6f42/users/305ec6dfacc943ea87b05a4dd7a2156b/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.540020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c14b1c27d4f43508818f2d87d4b6f42, user_id=305ec6dfacc943ea87b05a4dd7a2156b, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.542701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.554881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1321/2635] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:58 2022] PUT /identity/v3/projects/8c14b1c27d4f43508818f2d87d4b6f42/users/305ec6dfacc943ea87b05a4dd7a2156b/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:13:58.558726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.568369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b62a7-8124-4459-aa5e-2397d43e9d1c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.623317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea393f6-7145-4ef5-82a6-8d36937d9a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.623908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea393f6-7145-4ef5-82a6-8d36937d9a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.624183 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea393f6-7145-4ef5-82a6-8d36937d9a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.624450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea393f6-7145-4ef5-82a6-8d36937d9a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.625072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea393f6-7145-4ef5-82a6-8d36937d9a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.626662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea393f6-7145-4ef5-82a6-8d36937d9a44 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.636558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1315/2636] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:13:58.640542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.651900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b216a2-410e-43f4-85ab-be7daf487150 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.657039 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.657721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.658029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.658304 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] PATH_INFO: `/v3/projects/8c14b1c27d4f43508818f2d87d4b6f42/users/305ec6dfacc943ea87b05a4dd7a2156b/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.660220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c14b1c27d4f43508818f2d87d4b6f42, user_id=305ec6dfacc943ea87b05a4dd7a2156b, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.662887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.665172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:58.674505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1322/2637] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:13:58 2022] PUT /identity/v3/projects/8c14b1c27d4f43508818f2d87d4b6f42/users/305ec6dfacc943ea87b05a4dd7a2156b/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:58.686113 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.686398 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.686669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.711598 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:58.714513 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] MFA Rules not processed for user `305ec6dfacc943ea87b05a4dd7a2156b`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:58.716784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:58.782721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1316/2638] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:58.793136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:58.794867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:58.850822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e28c2e-8f51-43e5-8b70-67584edea326 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:58.854403 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af136d6e-be78-419b-b31d-651451ee5ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:58.854994 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af136d6e-be78-419b-b31d-651451ee5ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:58.855267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af136d6e-be78-419b-b31d-651451ee5ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:58.855535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af136d6e-be78-419b-b31d-651451ee5ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:58.864176 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af136d6e-be78-419b-b31d-651451ee5ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:58.865575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af136d6e-be78-419b-b31d-651451ee5ae7 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:58.875248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1323/2639] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:59.002432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:59.066462 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18dd01-7f1d-47c8-9dbe-2b60780054f7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:59.072119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:59.072758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:59.073035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:59.073307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] PATH_INFO: `/v3/users/5dd8d2e3e02b48e499dcaf78be1862df` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:59.074477 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd8d2e3e02b48e499dcaf78be1862df)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:59.076391 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:59.121781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd8d2e3e02b48e499dcaf78be1862df'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.129136 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd8d2e3e02b48e499dcaf78be1862df'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.129746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd8d2e3e02b48e499dcaf78be1862df'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.136734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd8d2e3e02b48e499dcaf78be1862df'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.137349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd8d2e3e02b48e499dcaf78be1862df'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.154042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1317/2640] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:59 2022] DELETE /identity/v3/users/5dd8d2e3e02b48e499dcaf78be1862df => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:59.158885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:59.159193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:59.159465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:59.178272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:13:59.179453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:59.241596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1324/2641] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:13:59.254349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:59.255818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:59.276036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:13:59.323131 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ce6ce-7f5d-443f-a801-4c0b29354582 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:59.326641 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:59.327236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:59.327510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:59.327777 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:59.336500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:59.338035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:59.347173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1318/2642] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:13:59.618551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:59.620011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:13:59.631992 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d5560-9d70-4298-b391-8d3b33ad704f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:59.636653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:59.637276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:59.637551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:59.638090 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] PATH_INFO: `/v3/projects/a79c2381a7c8414a8f91150076e51a22` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:59.639181 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79c2381a7c8414a8f91150076e51a22)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:59.641108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:59.810677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79c2381a7c8414a8f91150076e51a22', 'request_id': 'req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.816070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79c2381a7c8414a8f91150076e51a22', 'request_id': 'req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.825289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79c2381a7c8414a8f91150076e51a22', 'request_id': 'req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.838542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] The token cache is being invalidate because project a79c2381a7c8414a8f91150076e51a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:13:59.838892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.839533 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.841228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1325/2643] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:13:59 2022] DELETE /identity/v3/projects/a79c2381a7c8414a8f91150076e51a22 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:13:59.845203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:13:59.856163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b28bb-764a-4fa4-a019-6eae455d1759 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:13:59.915905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:13:59.916467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:13:59.916745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:13:59.917046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] PATH_INFO: `/v3/users/305ec6dfacc943ea87b05a4dd7a2156b` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:13:59.918241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=305ec6dfacc943ea87b05a4dd7a2156b)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:13:59.920377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:13:59.968785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305ec6dfacc943ea87b05a4dd7a2156b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.976071 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305ec6dfacc943ea87b05a4dd7a2156b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.976754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305ec6dfacc943ea87b05a4dd7a2156b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.983768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305ec6dfacc943ea87b05a4dd7a2156b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:13:59.984400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dfb09ab-5c97-46f4-a23d-fbbd62a30a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305ec6dfacc943ea87b05a4dd7a2156b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:00.002568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1319/2644] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:13:59 2022] DELETE /identity/v3/users/305ec6dfacc943ea87b05a4dd7a2156b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:00.274672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.276111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:00.289560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f76acb-c81c-4a1a-a9a6-08b044c4bba0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.306058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:00.349433 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.350094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.350422 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.350692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] PATH_INFO: `/v3/projects/8c14b1c27d4f43508818f2d87d4b6f42` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.351790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c14b1c27d4f43508818f2d87d4b6f42)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.354432 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.414962 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c14b1c27d4f43508818f2d87d4b6f42', 'request_id': 'req-7bd311e0-40c8-4995-b8e7-2cceec3066b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:00.421107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c14b1c27d4f43508818f2d87d4b6f42', 'request_id': 'req-7bd311e0-40c8-4995-b8e7-2cceec3066b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:00.429675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c14b1c27d4f43508818f2d87d4b6f42', 'request_id': 'req-7bd311e0-40c8-4995-b8e7-2cceec3066b6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:00.442313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] The token cache is being invalidate because project 8c14b1c27d4f43508818f2d87d4b6f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:00.442904 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:00.443569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7bd311e0-40c8-4995-b8e7-2cceec3066b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:00.445282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1326/2645] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:00 2022] DELETE /identity/v3/projects/8c14b1c27d4f43508818f2d87d4b6f42 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:00.450564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.451089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.451235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.473564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:00.474857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:00.508092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:00.536146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1320/2646] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:00.544283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.544660 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.544919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.563750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:00.564930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:00.573145 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1327/2647] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:00.577261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.586774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a7a98-3ada-4251-8824-b0eb4be7326a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.591905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.592534 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.592809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.593203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.593850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:00.594181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.595758 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.607165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1321/2648] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:14:00.611389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.620340 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e77ae-8818-496f-b4ca-d7bce7ce20fa None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.624901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.625483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.625759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.626063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.626680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.628549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.656808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1328/2649] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:00 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:00.660841 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.670827 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c8231-38d4-4fe9-9ae0-328d8205150e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.727303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.727849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.728152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.728453 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.729076 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.731060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.752645 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1322/2650] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:00 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:00.756811 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.768006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fdb32-6f82-4f9f-8bff-8ce827f88395 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.773103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.773742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.774077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.774351 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.774980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.777190 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.787596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1329/2651] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:00.791748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.802217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94bd66-fb14-4020-8ce3-2c08d4346e57 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.806823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ff5dd-7460-463c-aa96-f735b2a4157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.807418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff5dd-7460-463c-aa96-f735b2a4157f admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.807696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff5dd-7460-463c-aa96-f735b2a4157f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.807964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ff5dd-7460-463c-aa96-f735b2a4157f admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.808594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff5dd-7460-463c-aa96-f735b2a4157f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.810211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ff5dd-7460-463c-aa96-f735b2a4157f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.819970 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1323/2652] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:00.824521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:00.835652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13872d-0ad6-4cbc-9c47-a48236bddd8f admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:00.841384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:00.842209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.842475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.842792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] PATH_INFO: `/v3/projects/ae5dbbffea274acab163c2797f557f4d/users/65c19359a76e4010b777ce3e84122c17/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.857361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5dbbffea274acab163c2797f557f4d, user_id=65c19359a76e4010b777ce3e84122c17, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:00.859996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:00.872053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1330/2653] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:00 2022] PUT /identity/v3/projects/ae5dbbffea274acab163c2797f557f4d/users/65c19359a76e4010b777ce3e84122c17/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:00.883212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:00.883500 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:00.883775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:00.913582 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] MFA Rules not processed for user `65c19359a76e4010b777ce3e84122c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:00.914985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:00.984157 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1324/2654] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:00.989014 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.035966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:01.051865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f77fd0-d298-4f25-9f1e-a01fec2c81e7 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.057286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.058059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.058347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.058615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.059253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.061177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.088315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1331/2655] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:00 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:01.092430 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.102958 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39eb03-ae07-4503-92c4-7f7416c41db0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.135587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:01.161026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.161633 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.161905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.162207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.162825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.164716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.185823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1325/2656] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:01 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:01.189792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.200390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da602e0d-af6c-406f-b17f-06e0181cf099 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.204872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.205482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.205758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.206055 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.206682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.208274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.228127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1332/2657] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:01.232434 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.243689 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191511dc-e419-46b5-bffb-d6fa55380d0e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.250020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.250995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.251422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.251817 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] PATH_INFO: `/v3/projects/71f1b542ebf740fcad6cf2ab13a16abc/users/f23828f461724b90a6d9c6f63d2b1cc5/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.273570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f1b542ebf740fcad6cf2ab13a16abc, user_id=f23828f461724b90a6d9c6f63d2b1cc5, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.277907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.293339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1326/2658] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:01 2022] PUT /identity/v3/projects/71f1b542ebf740fcad6cf2ab13a16abc/users/f23828f461724b90a6d9c6f63d2b1cc5/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:01.297231 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.307902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9103de6-9455-41db-8aa2-714854260e8e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.362647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51bcd3-1d89-4aca-b6a6-dfc527e47ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.363001 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51bcd3-1d89-4aca-b6a6-dfc527e47ead admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.363282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51bcd3-1d89-4aca-b6a6-dfc527e47ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.363553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51bcd3-1d89-4aca-b6a6-dfc527e47ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.364639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51bcd3-1d89-4aca-b6a6-dfc527e47ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.365765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba51bcd3-1d89-4aca-b6a6-dfc527e47ead admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.376547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1333/2659] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:01.380816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.391657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e5cf2-9ac4-4515-a2bb-cb7299af3c09 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.399175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.400295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.400802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.401291 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] PATH_INFO: `/v3/projects/71f1b542ebf740fcad6cf2ab13a16abc/users/f23828f461724b90a6d9c6f63d2b1cc5/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.405102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f1b542ebf740fcad6cf2ab13a16abc, user_id=f23828f461724b90a6d9c6f63d2b1cc5, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.410270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.427982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1327/2660] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:01 2022] PUT /identity/v3/projects/71f1b542ebf740fcad6cf2ab13a16abc/users/f23828f461724b90a6d9c6f63d2b1cc5/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:01.438599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.438879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.439149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.469041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] MFA Rules not processed for user `f23828f461724b90a6d9c6f63d2b1cc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:01.470296 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:01.541931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1334/2661] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:01.553185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.554912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:01.617849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160e0e3-5a26-4834-8eb8-14c11004ae10 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.622064 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b9c052-667f-4f4a-beb2-3cd54fdcf692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.622667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c052-667f-4f4a-beb2-3cd54fdcf692 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.622939 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c052-667f-4f4a-beb2-3cd54fdcf692 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.623207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b9c052-667f-4f4a-beb2-3cd54fdcf692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.632276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9c052-667f-4f4a-beb2-3cd54fdcf692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.633815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b9c052-667f-4f4a-beb2-3cd54fdcf692 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.644047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1328/2662] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:01.725266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:01.740228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7462b7-976c-4d1d-a869-cb00ba1fe3a9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:01.744306 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:01.744908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:01.745238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:01.745456 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:01.794103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:01.813780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:01.815249 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:01.824816 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1335/2663] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:07.147627 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595b93d-6c83-4fe0-a589-f7540c1d152f None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:07.148204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595b93d-6c83-4fe0-a589-f7540c1d152f None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:07.148204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595b93d-6c83-4fe0-a589-f7540c1d152f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:07.168498 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e595b93d-6c83-4fe0-a589-f7540c1d152f None None] MFA Rules not processed for user `60edcebcccaa4da99ffddadd3e27c1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:07.169695 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e595b93d-6c83-4fe0-a589-f7540c1d152f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:07.227774 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1329/2664] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:10.804548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:10.866553 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a50afb-ff25-49eb-8a3c-fa8cd1e6a0dc service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:10.871587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:10.872205 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:10.872483 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:10.872752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] PATH_INFO: `/v3/users/65c19359a76e4010b777ce3e84122c17` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:10.873892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c19359a76e4010b777ce3e84122c17)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:10.875817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:10.919526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c19359a76e4010b777ce3e84122c17'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:10.925983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c19359a76e4010b777ce3e84122c17'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:10.926651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c19359a76e4010b777ce3e84122c17'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:10.927250 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c19359a76e4010b777ce3e84122c17'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:10.934627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c19359a76e4010b777ce3e84122c17'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:10.951708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1336/2665] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:10 2022] DELETE /identity/v3/users/65c19359a76e4010b777ce3e84122c17 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:10.956197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:10.956676 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:10.956823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:10.975161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:10.976358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:11.005318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:11.034858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1330/2666] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:11.045115 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:11.046541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:11.104616 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a8a29-90a3-4a90-8299-faa011b4c9e8 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:11.108314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:11.108921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:11.109199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:11.109473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:11.118535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:11.120529 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:11.129796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1337/2667] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:11.401549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:11.403054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:11.419412 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a4789-ad50-4807-8f85-ca4c8a99e141 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:11.424140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:11.424807 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:11.425087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:11.425360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] PATH_INFO: `/v3/projects/ae5dbbffea274acab163c2797f557f4d` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:11.426530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae5dbbffea274acab163c2797f557f4d)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:11.428480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:11.477778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae5dbbffea274acab163c2797f557f4d', 'request_id': 'req-054277df-0631-45ea-8bcc-4cbf5bf29b37'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.485974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae5dbbffea274acab163c2797f557f4d', 'request_id': 'req-054277df-0631-45ea-8bcc-4cbf5bf29b37'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.492448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae5dbbffea274acab163c2797f557f4d', 'request_id': 'req-054277df-0631-45ea-8bcc-4cbf5bf29b37'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.503916 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] The token cache is being invalidate because project ae5dbbffea274acab163c2797f557f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:11.504203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.504850 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.506355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1331/2668] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:11 2022] DELETE /identity/v3/projects/ae5dbbffea274acab163c2797f557f4d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:11.510233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:11.514953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:11.520735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109f3fe-a287-4072-8613-c5e5565c58ae service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:11.577116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:11.577762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:11.578120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:11.578399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] PATH_INFO: `/v3/users/f23828f461724b90a6d9c6f63d2b1cc5` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:11.579525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=f23828f461724b90a6d9c6f63d2b1cc5)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:11.581419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:11.626804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f23828f461724b90a6d9c6f63d2b1cc5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.633891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f23828f461724b90a6d9c6f63d2b1cc5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.634530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f23828f461724b90a6d9c6f63d2b1cc5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.635260 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f23828f461724b90a6d9c6f63d2b1cc5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.642508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-b8a0d85d-4b3e-4104-a061-1045fb1114ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f23828f461724b90a6d9c6f63d2b1cc5'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:11.663960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1338/2669] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:11 2022] DELETE /identity/v3/users/f23828f461724b90a6d9c6f63d2b1cc5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:12.160232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.161919 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:12.183595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054277df-0631-45ea-8bcc-4cbf5bf29b37 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.249822 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.250501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.250780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.251051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] PATH_INFO: `/v3/projects/71f1b542ebf740fcad6cf2ab13a16abc` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.252193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f1b542ebf740fcad6cf2ab13a16abc)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.254146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.304223 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f1b542ebf740fcad6cf2ab13a16abc', 'request_id': 'req-48926caf-6199-4f0b-a659-4330585823b7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:12.313702 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f1b542ebf740fcad6cf2ab13a16abc', 'request_id': 'req-48926caf-6199-4f0b-a659-4330585823b7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:12.320043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:12.323382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f1b542ebf740fcad6cf2ab13a16abc', 'request_id': 'req-48926caf-6199-4f0b-a659-4330585823b7'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:12.346316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] The token cache is being invalidate because project 71f1b542ebf740fcad6cf2ab13a16abc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:12.346591 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:12.347435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-48926caf-6199-4f0b-a659-4330585823b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:12.349140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1332/2670] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:12 2022] DELETE /identity/v3/projects/71f1b542ebf740fcad6cf2ab13a16abc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:12.354715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.355281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.355281 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.380436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:12.381832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:12.458536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1339/2671] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:12.467370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.468051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.468388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.496195 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:12.497448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:12.507351 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1333/2672] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:12.512931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.525419 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb11af-ba72-4240-aeac-bce8f3a91a64 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.534614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.535987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.536271 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.536546 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.537206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:12.537509 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.539622 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.555348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1340/2673] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:12.560537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.570510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0474fb-5f9d-439f-96db-008d94b7adc1 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.575703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.576429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.576791 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.577122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.577833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.579859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.618030 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1334/2674] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:12 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:12.620862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.631935 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a959a9f-841d-4da1-bfc6-6c306d2ac283 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.639255 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:12.706709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.707444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.707791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.708133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.708845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.711002 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.742652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1341/2675] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:12 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:12.743749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.757353 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ae8d2-4370-428f-a1bb-68e24ea3d56f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.762925 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.763632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.764013 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.764356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.765079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.766800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.783862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1335/2676] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:12.787412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.803950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f467e2-139c-45c4-878a-185baa8f48bf admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.810248 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd8a32-4df1-46c5-bcc9-d6a866e9f885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.811046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd8a32-4df1-46c5-bcc9-d6a866e9f885 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.811331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd8a32-4df1-46c5-bcc9-d6a866e9f885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.811604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd8a32-4df1-46c5-bcc9-d6a866e9f885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.812258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd8a32-4df1-46c5-bcc9-d6a866e9f885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.813858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd8a32-4df1-46c5-bcc9-d6a866e9f885 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.825946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1342/2677] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:12.831142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:12.841880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab186f-ac70-4863-a1b7-1bf436c60262 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:12.848002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:12.848002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.848002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.848002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] PATH_INFO: `/v3/projects/db48bc0bc6534e3ba4a9fa32aca55da6/users/00493a09f0ca44179b75ccdc5ffffc65/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.864632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db48bc0bc6534e3ba4a9fa32aca55da6, user_id=00493a09f0ca44179b75ccdc5ffffc65, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:12.867448 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:12.883727 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1336/2678] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:12 2022] PUT /identity/v3/projects/db48bc0bc6534e3ba4a9fa32aca55da6/users/00493a09f0ca44179b75ccdc5ffffc65/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:12.952425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:12.952614 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:12.952884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:12.990094 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] MFA Rules not processed for user `00493a09f0ca44179b75ccdc5ffffc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:12.991324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:13.062316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1343/2679] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:13.067273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.132530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a48ab4-3c79-4987-b942-c0377a565181 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.137922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.138659 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.139014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.139362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.140089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.142031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.176982 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1337/2680] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:13 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:13.180928 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.193284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49eecc-6404-49b8-9955-ac56ed793d1a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.268684 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.269282 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.269558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.269825 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.270474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.272364 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.294739 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1344/2681] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:13 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:13.298309 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.309229 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e416de-b8f7-4256-a6b6-f9af92f284be admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.314945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.315746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.316188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.316639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.317670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.320731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.334203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1338/2682] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:13.339626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.351045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669a939-3264-4251-a748-4b89a04fc838 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.358324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.359872 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.360152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.360428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] PATH_INFO: `/v3/projects/bb36d84c435b4dbe8d9167e440398fc7/users/e59bc0f44fdd4637aede9d49c035dab6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.363087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:13.380885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb36d84c435b4dbe8d9167e440398fc7, user_id=e59bc0f44fdd4637aede9d49c035dab6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.383585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.401828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1345/2683] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:13 2022] PUT /identity/v3/projects/bb36d84c435b4dbe8d9167e440398fc7/users/e59bc0f44fdd4637aede9d49c035dab6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:13.406646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.418974 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183351a-0b8e-466e-8d5f-49aed0bed499 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.487054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e7fb3-8b22-4a59-885a-a6ea14179759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.487753 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e7fb3-8b22-4a59-885a-a6ea14179759 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.488097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e7fb3-8b22-4a59-885a-a6ea14179759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.488432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e7fb3-8b22-4a59-885a-a6ea14179759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.489152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e7fb3-8b22-4a59-885a-a6ea14179759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.490993 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e7fb3-8b22-4a59-885a-a6ea14179759 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.492511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:13.506748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.507176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1339/2684] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:13.517868 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30bf22-ea8c-49c7-9754-b7d7966c87f2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.522550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.523197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.523469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.523735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] PATH_INFO: `/v3/projects/bb36d84c435b4dbe8d9167e440398fc7/users/e59bc0f44fdd4637aede9d49c035dab6/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.526275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb36d84c435b4dbe8d9167e440398fc7, user_id=e59bc0f44fdd4637aede9d49c035dab6, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.528882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.541139 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1346/2685] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:13 2022] PUT /identity/v3/projects/bb36d84c435b4dbe8d9167e440398fc7/users/e59bc0f44fdd4637aede9d49c035dab6/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:13.551438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.551723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.552020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.614740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] MFA Rules not processed for user `e59bc0f44fdd4637aede9d49c035dab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:13.615984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:13.742799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1340/2686] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:13.753523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:13.756947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:13.840086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f9856-cd25-43fd-b511-538717dc2f60 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:13.844034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2a68e-8a54-4348-a553-48d3f5db072c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:13.844741 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2a68e-8a54-4348-a553-48d3f5db072c service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:13.845087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2a68e-8a54-4348-a553-48d3f5db072c service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:13.845571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2a68e-8a54-4348-a553-48d3f5db072c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:13.856050 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2a68e-8a54-4348-a553-48d3f5db072c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:13.857688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2a68e-8a54-4348-a553-48d3f5db072c service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:13.869653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1347/2687] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:14.710831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:14.811899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c440bec-67bf-4f36-b35e-8d3bcb38376b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:14.819950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:14.820667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:14.821167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:14.821681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] PATH_INFO: `/v3/users/00493a09f0ca44179b75ccdc5ffffc65` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:14.824396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00493a09f0ca44179b75ccdc5ffffc65)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:14.826832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:14.907683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00493a09f0ca44179b75ccdc5ffffc65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:14.918744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00493a09f0ca44179b75ccdc5ffffc65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:14.919975 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00493a09f0ca44179b75ccdc5ffffc65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:14.930199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00493a09f0ca44179b75ccdc5ffffc65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:14.932520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00493a09f0ca44179b75ccdc5ffffc65'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:14.957026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:14.967569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1341/2688] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:14 2022] DELETE /identity/v3/users/00493a09f0ca44179b75ccdc5ffffc65 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:14.980417 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:14.981770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:14.982287 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:15.008688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:15.010377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:15.096911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1348/2689] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:15.123482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:15.125149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:15.219165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2072a-0a2e-4bc4-a518-02fd7984fa26 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:15.223526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:15.224277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:15.224625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:15.224969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:15.238154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:15.239711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:15.262648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1342/2690] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:16.121605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:16.123179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:16.134884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ba4f2-4cd6-4916-a79e-d39e594daf0f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:16.139517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:16.140147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:16.140423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:16.140692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] PATH_INFO: `/v3/projects/db48bc0bc6534e3ba4a9fa32aca55da6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:16.142026 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db48bc0bc6534e3ba4a9fa32aca55da6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:16.144224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:16.224879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db48bc0bc6534e3ba4a9fa32aca55da6', 'request_id': 'req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.233700 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db48bc0bc6534e3ba4a9fa32aca55da6', 'request_id': 'req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.247154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db48bc0bc6534e3ba4a9fa32aca55da6', 'request_id': 'req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.258027 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:16.271716 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:16.273864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:16.274490 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] The token cache is being invalidate because project db48bc0bc6534e3ba4a9fa32aca55da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:16.274951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.276334 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.278734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1349/2691] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:16 2022] DELETE /identity/v3/projects/db48bc0bc6534e3ba4a9fa32aca55da6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:16.282602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:16.285129 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:16.394320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f46b7-f3c6-4093-a53d-39bd020e4c1c admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:16.399256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6ee04-9509-4361-9d74-26407dc9ee79 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:16.411662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:16.412799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:16.414522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:16.414923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] PATH_INFO: `/v3/users/e59bc0f44fdd4637aede9d49c035dab6` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:16.422890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59bc0f44fdd4637aede9d49c035dab6)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:16.425369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:16.520976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59bc0f44fdd4637aede9d49c035dab6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.546307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59bc0f44fdd4637aede9d49c035dab6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.547517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59bc0f44fdd4637aede9d49c035dab6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.548320 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59bc0f44fdd4637aede9d49c035dab6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.560163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:16.560705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:16.561110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:16.561385 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:16.563849 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:16.563993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59bc0f44fdd4637aede9d49c035dab6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:16.578087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:16.581043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c9588-73af-4cfb-b76f-bd68a1187b68 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:16.588400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1350/2692] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:16 2022] DELETE /identity/v3/users/e59bc0f44fdd4637aede9d49c035dab6 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:16.593565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1343/2693] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:16.981354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:16.983235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:17.092471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e6e68-48c8-4ba6-a2f4-c57922fd93ba admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.101921 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.102717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.103589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.103950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] PATH_INFO: `/v3/projects/bb36d84c435b4dbe8d9167e440398fc7` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.105238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb36d84c435b4dbe8d9167e440398fc7)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.108004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.129726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:17.184236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb36d84c435b4dbe8d9167e440398fc7', 'request_id': 'req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:17.193722 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb36d84c435b4dbe8d9167e440398fc7', 'request_id': 'req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:17.223444 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb36d84c435b4dbe8d9167e440398fc7', 'request_id': 'req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:17.246648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] The token cache is being invalidate because project bb36d84c435b4dbe8d9167e440398fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:17.246993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:17.247637 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ba81a61e-6edf-4b5c-ad0d-cd268133be8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:17.252866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1351/2694] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:16 2022] DELETE /identity/v3/projects/bb36d84c435b4dbe8d9167e440398fc7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:17.254536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.254985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.255440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.292720 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:17.294615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:17.330270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=100 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:17.414304 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1344/2695] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:17.421788 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.422118 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.422388 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.462363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:17.463854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:17.478791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1352/2696] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:17.481040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:17.492629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663dbe2f-1656-4143-b5e4-a3c0c9ab0948 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.499450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.500164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.500717 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.501255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.502163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:17.502746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.504724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.518457 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1345/2697] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:17.522767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:17.540850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef61f3-4374-49c1-8868-c35c9a0c7cc7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.548428 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.549280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.549725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.551155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.551155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.552974 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.596219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1353/2698] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:17 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:17.599973 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:17.611895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0b006-104a-49be-8ec3-bfba8d38bee1 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.673873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.675171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.675556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.675832 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.676559 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.679333 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.705789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1346/2699] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:17 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:17.709891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:17.721854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e8ae2-ceb9-4b8a-9993-8efd87036388 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.727648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.728393 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.728939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.729346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.730134 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.732087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.744789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1354/2700] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:17.749359 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:17.765715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4ed6a-e663-4f6e-9732-1f330ac3e782 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.772103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fae4c-1a62-4320-bc12-12da61e035bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.772869 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fae4c-1a62-4320-bc12-12da61e035bd admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.773846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fae4c-1a62-4320-bc12-12da61e035bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.775634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fae4c-1a62-4320-bc12-12da61e035bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.775634 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fae4c-1a62-4320-bc12-12da61e035bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.777444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fae4c-1a62-4320-bc12-12da61e035bd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.794140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1347/2701] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:17.797211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:17.812817 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9183faf-fdc1-4dbe-8671-cfa08458fd1d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:17.819427 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:17.820100 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.820383 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.820690 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] PATH_INFO: `/v3/projects/1b9784dd258c44ca93dda990a436a054/users/76d39d666185491db259b4ce40a6dfe8/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.850151 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9784dd258c44ca93dda990a436a054, user_id=76d39d666185491db259b4ce40a6dfe8, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:17.852915 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:17.876628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1355/2702] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:17 2022] PUT /identity/v3/projects/1b9784dd258c44ca93dda990a436a054/users/76d39d666185491db259b4ce40a6dfe8/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:17.888219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:17.888515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:17.888799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:17.922837 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] MFA Rules not processed for user `76d39d666185491db259b4ce40a6dfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:17.925120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:18.018460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1348/2703] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:18.019860 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.100775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bac882-c043-4ead-abd8-cf5ff0a84b8d admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.107332 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.108127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.110037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.110439 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.112153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.113380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.144472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1356/2704] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:18 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:18.148642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.160523 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471d457-23e3-4afd-909c-4b56fe5ea3b6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.179483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:18.220300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.220908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.221182 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.221442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.222175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.224077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.247519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1349/2705] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:18 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:18.251680 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.265557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd841f5-e52f-4da4-ac63-d43c45902d80 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.271543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.273047 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.273514 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.274029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.274840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.277305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.289288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1357/2706] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:18.294785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.305178 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25e8f-9098-44a0-bd88-b81a3927099a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.310254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f45133-c651-4753-8211-5f46f104c0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.310842 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45133-c651-4753-8211-5f46f104c0fc admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.311124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45133-c651-4753-8211-5f46f104c0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.311396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f45133-c651-4753-8211-5f46f104c0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.312058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f45133-c651-4753-8211-5f46f104c0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.313690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f45133-c651-4753-8211-5f46f104c0fc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.323794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1350/2707] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:18.327956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.338311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cad006-5942-4dff-a0db-8bb2ec276d65 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.342960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.343652 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.343927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.344197 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] PATH_INFO: `/v3/projects/70f12b01d97f4b108845a810ad71296f/users/e05d2628c2ec45328eb748e725c06d95/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.356395 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:18.359787 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f12b01d97f4b108845a810ad71296f, user_id=e05d2628c2ec45328eb748e725c06d95, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.363356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.378380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1358/2708] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:18 2022] PUT /identity/v3/projects/70f12b01d97f4b108845a810ad71296f/users/e05d2628c2ec45328eb748e725c06d95/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:18.388870 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.389162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.389445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.419786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] MFA Rules not processed for user `e05d2628c2ec45328eb748e725c06d95`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:18.421104 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:18.494978 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1351/2709] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:18.499387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.571354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42396df-9f16-4780-97f8-2ae8cc5eaf11 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.576602 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.577207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.577491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.577765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.578431 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.580331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.610748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1359/2710] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:18 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:18.615004 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.628185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac9404-d613-4452-a474-4401dd7558b2 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.695265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.695883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.696164 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.696442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.697595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.699544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.722171 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1352/2711] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:18 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:18.726357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.738919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276af5b-950e-4e6a-89e6-eb4460daa4bf admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.742802 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.743455 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.743858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.744257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.744902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.746527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.756916 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1360/2712] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:18.761561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.771694 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d8de27-f05f-461a-8d63-48dbbec65555 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.777009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.777652 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.778245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.778403 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] PATH_INFO: `/v3/projects/40cf3f0bd7fd46aeb02a1bc8d90ca9b2/users/805e22a3e5ba45afae89447a843cefa1/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.793762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cf3f0bd7fd46aeb02a1bc8d90ca9b2, user_id=805e22a3e5ba45afae89447a843cefa1, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.796452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.799501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:18.812597 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1353/2713] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:18 2022] PUT /identity/v3/projects/40cf3f0bd7fd46aeb02a1bc8d90ca9b2/users/805e22a3e5ba45afae89447a843cefa1/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:18.816522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.827203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031d9ff-15c6-40bc-afe9-690916c81daf admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.884951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a7b4c-4505-47d0-ba54-0eec4ae023cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.885554 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7b4c-4505-47d0-ba54-0eec4ae023cb admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.885982 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7b4c-4505-47d0-ba54-0eec4ae023cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.886291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7b4c-4505-47d0-ba54-0eec4ae023cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.886945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a7b4c-4505-47d0-ba54-0eec4ae023cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.888581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a7b4c-4505-47d0-ba54-0eec4ae023cb admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.899993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1361/2714] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:18.904308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:18.915093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19778-5f4c-440d-b3a6-f4ec033cad75 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:18.919772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:18.920450 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.920726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.920997 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] PATH_INFO: `/v3/projects/40cf3f0bd7fd46aeb02a1bc8d90ca9b2/users/805e22a3e5ba45afae89447a843cefa1/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.923099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cf3f0bd7fd46aeb02a1bc8d90ca9b2, user_id=805e22a3e5ba45afae89447a843cefa1, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:18.925799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:18.938078 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1354/2715] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:18 2022] PUT /identity/v3/projects/40cf3f0bd7fd46aeb02a1bc8d90ca9b2/users/805e22a3e5ba45afae89447a843cefa1/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:18.950504 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:18.950880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:18.951246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:18.981627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] MFA Rules not processed for user `805e22a3e5ba45afae89447a843cefa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:18.982888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:19.034534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:19.059568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1362/2716] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:19.071628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:19.073985 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:19.144724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec692f-a661-48c6-ac3d-a2bd70e2821a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:19.149759 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:19.150431 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:19.150710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:19.151520 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:19.222234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:19.223677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:19.233422 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1355/2717] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:19.848530 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:19.849913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:19.905955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb815de-26a0-4666-801a-706452cf43ef None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:19.909446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:19.910067 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:19.910349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:19.910621 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:19.912297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:19.973301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:19.974742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:19.984275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1363/2718] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:20.798324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:20.807480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dcc83-72ee-4623-aac0-41344f4f7ed3 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:20.811221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b660791d-bfbd-464a-9eae-131012c35a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:20.811851 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660791d-bfbd-464a-9eae-131012c35a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:20.812133 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660791d-bfbd-464a-9eae-131012c35a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:20.812404 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b660791d-bfbd-464a-9eae-131012c35a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:20.876203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660791d-bfbd-464a-9eae-131012c35a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:20.878679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b660791d-bfbd-464a-9eae-131012c35a0c service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:20.889724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1356/2719] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:21.457796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:21.468179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f15ad3-959c-404c-8a3c-27bd8e325244 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:21.472654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:21.473303 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:21.474283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:21.474283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:21.485171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:21.486789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:21.496654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1364/2720] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:23.115943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:23.116480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:23.116480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:23.138614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:23.140021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:23.182283 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:23.222203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1357/2721] 10.222.0.50 () {58 vars in 967 bytes} [Sat Sep 24 05:14:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:23.280548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:23.282147 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:23.393172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336031b-308e-49ff-b869-d0285114d6a6 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:23.406809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:23.408869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:23.409940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:23.410339 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:23.420526 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:23.422166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:23.434183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1365/2722] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:14:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:14:23.648746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:23.650362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:23.669368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f0fc-1cf9-4bcf-b7fa-885ee4853f1a None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:23.675062 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:23.675658 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:23.676029 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:23.676467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] PATH_INFO: `/v3/users/74a8cc19ab204424ab23c62915ac2bab` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:23.677733 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a8cc19ab204424ab23c62915ac2bab)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:23.680079 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:23.730797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a8cc19ab204424ab23c62915ac2bab'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:23.739954 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a8cc19ab204424ab23c62915ac2bab'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:23.740776 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a8cc19ab204424ab23c62915ac2bab'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:23.750863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a8cc19ab204424ab23c62915ac2bab'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:23.751612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8b720dec-7f5d-4346-ba58-0bb6f5d4d509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a8cc19ab204424ab23c62915ac2bab'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:23.769316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1358/2723] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:23 2022] DELETE /identity/v3/users/74a8cc19ab204424ab23c62915ac2bab => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:23.949218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:23.961931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef592b1f-ec40-403f-995f-906061848227 service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:23.980149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:24.053883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.054496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.054773 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.055046 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] PATH_INFO: `/v3/projects/87d5229e39fe4817b5187068d8acecc7` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.056995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d5229e39fe4817b5187068d8acecc7)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.058913 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.131576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d5229e39fe4817b5187068d8acecc7', 'request_id': 'req-56a5a5fd-3abd-4596-a677-535790abee94'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:24.138077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d5229e39fe4817b5187068d8acecc7', 'request_id': 'req-56a5a5fd-3abd-4596-a677-535790abee94'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:24.146884 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d5229e39fe4817b5187068d8acecc7', 'request_id': 'req-56a5a5fd-3abd-4596-a677-535790abee94'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:24.163387 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] The token cache is being invalidate because project 87d5229e39fe4817b5187068d8acecc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:24.163677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:24.166555 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-56a5a5fd-3abd-4596-a677-535790abee94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:24.168476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1366/2724] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:23 2022] DELETE /identity/v3/projects/87d5229e39fe4817b5187068d8acecc7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:24.173264 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.173752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.173854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.199307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:24.200518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:24.272555 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1359/2725] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:24.288627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.289218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.289218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.311168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:24.312612 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:24.320778 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1367/2726] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:24.325090 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.334857 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842554b-800a-4f25-b51d-c1e658c99983 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.340395 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.341015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.341297 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.341567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.342275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:24.342576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.344172 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.357743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1360/2727] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:24.362956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.372480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b45706-1ae0-419a-b489-8d751a8c3b84 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.378674 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.379396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.379588 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.380052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.380952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.383413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.422024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1368/2728] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/projects => generated 353 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:24.423142 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.436197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a2188-6075-483f-a220-b32ae93bdfac admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.477616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:24.501358 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.501979 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.502464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.502739 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.504630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.506616 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.531103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1361/2729] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/users => generated 277 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:24.535166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.546158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549944c-148b-4226-82f8-13da1a289504 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.550980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.551583 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.551867 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.552136 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.552771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.554400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.565262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1369/2730] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:24.569432 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.580188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a87e0-bdb7-44bd-8578-f3e1debf9fe1 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.585701 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db147f-10d8-485f-b3e3-5095f9bad10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.586364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db147f-10d8-485f-b3e3-5095f9bad10c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.586624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db147f-10d8-485f-b3e3-5095f9bad10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.586953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db147f-10d8-485f-b3e3-5095f9bad10c admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.588160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db147f-10d8-485f-b3e3-5095f9bad10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.589813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db147f-10d8-485f-b3e3-5095f9bad10c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.602240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1362/2731] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:24.606543 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.616503 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0800458e-1ed2-45ac-a255-8c8c41ce0f6e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.621317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.621967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.622277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.622557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] PATH_INFO: `/v3/projects/1c8b67a4632b4f77886909f3d585c231/users/952db8328a5e4d40849f62aefde55b88/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.639759 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8b67a4632b4f77886909f3d585c231, user_id=952db8328a5e4d40849f62aefde55b88, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.642548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.663579 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1370/2732] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:24 2022] PUT /identity/v3/projects/1c8b67a4632b4f77886909f3d585c231/users/952db8328a5e4d40849f62aefde55b88/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:24.675439 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.675728 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.676003 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.705036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] MFA Rules not processed for user `952db8328a5e4d40849f62aefde55b88`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:24.706298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:24.773272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1363/2733] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:24.778024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.825493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:24.849399 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8e387-d71c-4912-a6a7-23263a026e97 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.854589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.855221 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.855497 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.855762 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.856397 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.858338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:24.889628 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1371/2734] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/projects => generated 357 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:24.893483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:24.907961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75d46a-eb2c-4a79-87de-ac1547918b51 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:24.978879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:24.979923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:24.980180 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:24.980449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:24.981089 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:24.983429 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.013094 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1364/2735] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:24 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:25.017070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.029256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95039-3b32-4f52-81d9-666a74ab4a2a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.034313 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.034924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.035434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.035705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.036354 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.038138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.049895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1372/2736] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:25.054501 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.066781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b2aa6-191c-49dd-9b49-feb1cc53b48e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.071785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.072509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.072789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.073060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] PATH_INFO: `/v3/projects/7b4cb5b4be7f48eaae4b7b1e7927eafb/users/6f3b2518c0f24524b572d80e1cd0452e/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.088476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4cb5b4be7f48eaae4b7b1e7927eafb, user_id=6f3b2518c0f24524b572d80e1cd0452e, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.091140 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.106262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1365/2737] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:25 2022] PUT /identity/v3/projects/7b4cb5b4be7f48eaae4b7b1e7927eafb/users/6f3b2518c0f24524b572d80e1cd0452e/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:25.110148 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.120009 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9da5-786e-4001-91d6-cf1f10e8884b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.179470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f53d73-0c1e-4851-8f38-27e7a6f2d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.179975 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f53d73-0c1e-4851-8f38-27e7a6f2d1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.180254 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f53d73-0c1e-4851-8f38-27e7a6f2d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.180525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f53d73-0c1e-4851-8f38-27e7a6f2d1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.181163 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f53d73-0c1e-4851-8f38-27e7a6f2d1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.182820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f53d73-0c1e-4851-8f38-27e7a6f2d1d2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.196006 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1373/2738] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:25.197445 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.207368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d5b20-efd9-494c-94fb-ae22906b9beb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.211950 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.213151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.213424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.213752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] PATH_INFO: `/v3/projects/7b4cb5b4be7f48eaae4b7b1e7927eafb/users/6f3b2518c0f24524b572d80e1cd0452e/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.215715 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4cb5b4be7f48eaae4b7b1e7927eafb, user_id=6f3b2518c0f24524b572d80e1cd0452e, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.218427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.225746 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:25.236216 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1366/2739] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:25 2022] PUT /identity/v3/projects/7b4cb5b4be7f48eaae4b7b1e7927eafb/users/6f3b2518c0f24524b572d80e1cd0452e/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:25.241909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.242227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.242505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.273695 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] MFA Rules not processed for user `6f3b2518c0f24524b572d80e1cd0452e`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:25.275623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:25.346159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1374/2740] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:25.357396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.359130 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:25.421307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161da5ee-7269-47a6-8c11-9965355decc7 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.425044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.425730 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.426032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.426332 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.436868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.438362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.475163 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1367/2741] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:25.556506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.558396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:25.585493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.588111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:25.602402 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febfc1b-66ae-489e-bf5a-101c872959eb service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.604694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:25.608734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33d6b4-9325-4f76-83cf-b52686c60f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.611912 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d6b4-9325-4f76-83cf-b52686c60f81 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.612474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d6b4-9325-4f76-83cf-b52686c60f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.613200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33d6b4-9325-4f76-83cf-b52686c60f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.638343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a1b0-8757-4002-bfae-ab9d7f25531f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:25.644122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:25.644958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.645524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.645966 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] PATH_INFO: `/v3/users/76d39d666185491db259b4ce40a6dfe8` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.647891 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d39d666185491db259b4ce40a6dfe8)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.650832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.699279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33d6b4-9325-4f76-83cf-b52686c60f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:25.701796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff33d6b4-9325-4f76-83cf-b52686c60f81 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:25.713780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1368/2742] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:25 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:25.719029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d39d666185491db259b4ce40a6dfe8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:25.729548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d39d666185491db259b4ce40a6dfe8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:25.730402 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d39d666185491db259b4ce40a6dfe8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:25.731216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d39d666185491db259b4ce40a6dfe8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:25.748744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d39d666185491db259b4ce40a6dfe8'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:25.784297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1375/2743] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:25 2022] DELETE /identity/v3/users/76d39d666185491db259b4ce40a6dfe8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:25.787646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:25.788128 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:25.788793 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:25.822468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:25.824275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:25.936212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1369/2744] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:25.945532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:25.947070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:26.011592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f0b59-6e48-4979-8b1d-90b804986792 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:26.015301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:26.015906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:26.016180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:26.016447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:26.028126 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:26.029590 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:26.043194 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1376/2745] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:26.328101 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:26.331889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:26.344650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d766d85-9cb5-465f-bcdc-661594467876 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:26.350068 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:26.350722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:26.351002 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:26.351277 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] PATH_INFO: `/v3/projects/1b9784dd258c44ca93dda990a436a054` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:26.352459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9784dd258c44ca93dda990a436a054)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:26.354934 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:26.404669 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:26.417532 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9784dd258c44ca93dda990a436a054', 'request_id': 'req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.427496 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9784dd258c44ca93dda990a436a054', 'request_id': 'req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.437471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9784dd258c44ca93dda990a436a054', 'request_id': 'req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.449862 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] The token cache is being invalidate because project 1b9784dd258c44ca93dda990a436a054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:26.450212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.450922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.452803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1370/2746] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:26 2022] DELETE /identity/v3/projects/1b9784dd258c44ca93dda990a436a054 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:26.456839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:26.469214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed31769-b438-46c1-9313-4b7afea32556 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:26.527398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:26.535144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:26.536169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:26.536654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:26.537110 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] PATH_INFO: `/v3/users/e05d2628c2ec45328eb748e725c06d95` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:26.538626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05d2628c2ec45328eb748e725c06d95)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:26.541756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:26.597482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05d2628c2ec45328eb748e725c06d95'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.605813 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05d2628c2ec45328eb748e725c06d95'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.606581 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05d2628c2ec45328eb748e725c06d95'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.607275 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05d2628c2ec45328eb748e725c06d95'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.623122 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05d2628c2ec45328eb748e725c06d95'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:26.644369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1377/2747] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:26 2022] DELETE /identity/v3/users/e05d2628c2ec45328eb748e725c06d95 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:27.034394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:27.038225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:27.068576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f889bd3-03fd-4d4f-bb18-4c77f1fa729c admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:27.179246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:27.179887 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:27.180162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:27.180436 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] PATH_INFO: `/v3/projects/70f12b01d97f4b108845a810ad71296f` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:27.182111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f12b01d97f4b108845a810ad71296f)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:27.184815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:27.293225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f12b01d97f4b108845a810ad71296f', 'request_id': 'req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.308267 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f12b01d97f4b108845a810ad71296f', 'request_id': 'req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.319491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f12b01d97f4b108845a810ad71296f', 'request_id': 'req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.344509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] The token cache is being invalidate because project 70f12b01d97f4b108845a810ad71296f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:27.345201 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.345573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.350232 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1371/2748] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:27 2022] DELETE /identity/v3/projects/70f12b01d97f4b108845a810ad71296f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:27.351903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:27.354785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:27.370326 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f7f07c-0dc3-40f0-a407-a7bce09fb58e admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:27.448699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:27.449414 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:27.450057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:27.450057 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] PATH_INFO: `/v3/users/805e22a3e5ba45afae89447a843cefa1` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:27.452071 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] RBAC: Authorizing `identity:delete_user(user_id=805e22a3e5ba45afae89447a843cefa1)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:27.453599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:27.522445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805e22a3e5ba45afae89447a843cefa1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.532596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805e22a3e5ba45afae89447a843cefa1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.533421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805e22a3e5ba45afae89447a843cefa1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.534603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805e22a3e5ba45afae89447a843cefa1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.543493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-081d2d91-197d-4c71-8b57-98e299d4b63d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805e22a3e5ba45afae89447a843cefa1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:27.556447 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:27.566035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1378/2749] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:27 2022] DELETE /identity/v3/users/805e22a3e5ba45afae89447a843cefa1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:27.856533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:27.858266 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:27.872152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7dbd2-f79c-44f6-8826-f5e2c3f4432a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:27.930021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:27.937959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:27.939107 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:27.939503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:27.939784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] PATH_INFO: `/v3/projects/40cf3f0bd7fd46aeb02a1bc8d90ca9b2` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:27.943368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cf3f0bd7fd46aeb02a1bc8d90ca9b2)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:27.945525 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.003241 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cf3f0bd7fd46aeb02a1bc8d90ca9b2', 'request_id': 'req-8077ff64-f8aa-4212-994f-f4ec7095978c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:28.013723 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cf3f0bd7fd46aeb02a1bc8d90ca9b2', 'request_id': 'req-8077ff64-f8aa-4212-994f-f4ec7095978c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:28.021247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cf3f0bd7fd46aeb02a1bc8d90ca9b2', 'request_id': 'req-8077ff64-f8aa-4212-994f-f4ec7095978c'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:28.037054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] The token cache is being invalidate because project 40cf3f0bd7fd46aeb02a1bc8d90ca9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:28.037512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:28.038356 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-8077ff64-f8aa-4212-994f-f4ec7095978c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:28.041663 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1372/2750] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:27 2022] DELETE /identity/v3/projects/40cf3f0bd7fd46aeb02a1bc8d90ca9b2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:28.046536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.046826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.047098 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.067585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:28.068926 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:28.140839 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1379/2751] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:28.142544 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.142828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.143097 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.167298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:28.169426 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:28.183754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1373/2752] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:28.184580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.194659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112fff3-eafd-48ca-bd2a-f5034d71e436 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.199482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.200236 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.200523 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.200783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.201434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:28.201736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.203366 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.218517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1380/2753] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:28.223119 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.233341 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d906202-e11d-4c8b-a1ac-d448254cdc4b None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.238552 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.239265 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.239719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.240085 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.240775 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.243734 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.281889 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1374/2754] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:28 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:28.286177 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.296719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa77f5-d820-4d47-8f07-68f2f6ec878a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.359756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.360551 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.360898 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.361238 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.361976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.364030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.390728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1381/2755] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:28 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:28.394845 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.408369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4477fb1-ae69-41ad-b885-16fcb636eb0a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.413945 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.414773 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.415083 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.415364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.416014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.417872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.429863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1375/2756] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:28.434085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.447374 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303b105-b4f7-41f3-9170-1b450d31a795 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.453144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe6263-60c3-44ad-b753-be248336110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.453823 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe6263-60c3-44ad-b753-be248336110f admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.454200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe6263-60c3-44ad-b753-be248336110f admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.454557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe6263-60c3-44ad-b753-be248336110f admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.455274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe6263-60c3-44ad-b753-be248336110f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.456943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe6263-60c3-44ad-b753-be248336110f admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.469564 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1382/2757] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:28.474486 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.492382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913293c9-db4c-42d5-9b3e-589e913ba98f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.497646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.498565 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.498910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.499200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] PATH_INFO: `/v3/projects/3c7dd358b27c46c39025fc95f3d56997/users/abf719b5d31043cc937217f4610e4190/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.518750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7dd358b27c46c39025fc95f3d56997, user_id=abf719b5d31043cc937217f4610e4190, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.521405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.548710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1376/2758] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:28 2022] PUT /identity/v3/projects/3c7dd358b27c46c39025fc95f3d56997/users/abf719b5d31043cc937217f4610e4190/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:28.559440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.560356 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.560840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.601820 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] MFA Rules not processed for user `abf719b5d31043cc937217f4610e4190`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:28.603517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:28.700317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:28.723720 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1383/2759] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:28.735380 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:28.739270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:28.841595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8c130-aba0-4688-954f-911e86323694 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:28.845318 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4a141-aaa5-4f1b-9148-7e0c0977c3e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:28.845937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a141-aaa5-4f1b-9148-7e0c0977c3e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:28.846245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a141-aaa5-4f1b-9148-7e0c0977c3e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:28.846512 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4a141-aaa5-4f1b-9148-7e0c0977c3e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:28.856206 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4a141-aaa5-4f1b-9148-7e0c0977c3e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:28.857957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4a141-aaa5-4f1b-9148-7e0c0977c3e3 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:28.869596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1377/2760] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:29.179053 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:29.181879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:29.294316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa210f3-685f-4fad-9da0-f557c012fb36 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:29.301862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:29.302585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:29.302869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:29.303152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:29.305731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:29.326074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:29.327883 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:29.339175 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1384/2761] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:30.376762 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:30.377345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:30.377480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:30.398488 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] MFA Rules not processed for user `abf719b5d31043cc937217f4610e4190`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:30.401047 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:30.407138 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1378/2762] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:31.287574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:31.289235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:31.366605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b9f65-7e59-4eb3-9bf0-8b75a2a9fe0b service nova] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:31.381803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:31.382763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:31.383331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:31.384323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] PATH_INFO: `/v3/users/f67fe549bae6447f8b24219becb3bf97` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:31.385770 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67fe549bae6447f8b24219becb3bf97)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:31.387750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:31.452821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67fe549bae6447f8b24219becb3bf97'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.471953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67fe549bae6447f8b24219becb3bf97'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.472960 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67fe549bae6447f8b24219becb3bf97'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.473866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67fe549bae6447f8b24219becb3bf97'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.484801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67fe549bae6447f8b24219becb3bf97'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.499235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:31.511076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1385/2763] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:31 2022] DELETE /identity/v3/users/f67fe549bae6447f8b24219becb3bf97 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:31.694446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:31.707174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b8afb-f6ea-4f34-b77a-564e0dede078 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:31.720203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:31.768521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:31.769272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:31.769587 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:31.769924 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] PATH_INFO: `/v3/projects/85c73f9a74da4079a54a503e0b42ca2e` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:31.771263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c73f9a74da4079a54a503e0b42ca2e)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:31.773346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:31.823150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c73f9a74da4079a54a503e0b42ca2e', 'request_id': 'req-76fc3742-5f9c-4303-afac-fe6eb352055a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.830487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c73f9a74da4079a54a503e0b42ca2e', 'request_id': 'req-76fc3742-5f9c-4303-afac-fe6eb352055a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.836257 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c73f9a74da4079a54a503e0b42ca2e', 'request_id': 'req-76fc3742-5f9c-4303-afac-fe6eb352055a'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.847876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] The token cache is being invalidate because project 85c73f9a74da4079a54a503e0b42ca2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:31.848165 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.848859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.850370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1379/2764] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:31 2022] DELETE /identity/v3/projects/85c73f9a74da4079a54a503e0b42ca2e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:31.854404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:31.855897 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:31.870186 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136c848-e4da-469c-a065-f3f16de538f2 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:31.932951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:31.934389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:31.934735 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:31.935133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] PATH_INFO: `/v3/users/b42b2508925a4f9d9e17ce310a03d6e0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:31.936598 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b42b2508925a4f9d9e17ce310a03d6e0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:31.938651 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:31.987599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b42b2508925a4f9d9e17ce310a03d6e0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.995973 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b42b2508925a4f9d9e17ce310a03d6e0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.996589 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b42b2508925a4f9d9e17ce310a03d6e0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:31.997204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b42b2508925a4f9d9e17ce310a03d6e0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.004493 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b42b2508925a4f9d9e17ce310a03d6e0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.024372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1386/2765] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:31 2022] DELETE /identity/v3/users/b42b2508925a4f9d9e17ce310a03d6e0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:32.385786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:32.387298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:32.453239 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:32.454756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:32.462653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc3742-5f9c-4303-afac-fe6eb352055a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:32.548537 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:32.549294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:32.549569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:32.549840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:32.552015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:32.565144 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47933-bdfc-409c-9df2-ae937aa24056 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:32.571539 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:32.582187 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:32.582484 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:32.582768 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] PATH_INFO: `/v3/projects/ca0079034881467cb46e90bd21519543` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:32.585379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:32.587372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca0079034881467cb46e90bd21519543)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:32.606268 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:32.616545 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:32.661437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:32.663405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:32.667466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca0079034881467cb46e90bd21519543', 'request_id': 'req-d0a1c7b8-9e74-42c3-a230-3c805fc58803'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.673845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca0079034881467cb46e90bd21519543', 'request_id': 'req-d0a1c7b8-9e74-42c3-a230-3c805fc58803'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.680719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1380/2766] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:32.686789 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca0079034881467cb46e90bd21519543', 'request_id': 'req-d0a1c7b8-9e74-42c3-a230-3c805fc58803'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.719521 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] The token cache is being invalidate because project ca0079034881467cb46e90bd21519543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:32.719834 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.720707 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d0a1c7b8-9e74-42c3-a230-3c805fc58803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:32.724959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1387/2767] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:32 2022] DELETE /identity/v3/projects/ca0079034881467cb46e90bd21519543 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:41.483036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:41.484510 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:41.497768 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e713ce-d999-4026-9b91-916576b8da30 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:41.554396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:41.555019 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:41.555293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:41.555567 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] PATH_INFO: `/v3/users/952db8328a5e4d40849f62aefde55b88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:41.556683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=952db8328a5e4d40849f62aefde55b88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:41.558665 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:41.605120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '952db8328a5e4d40849f62aefde55b88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:41.612070 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '952db8328a5e4d40849f62aefde55b88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:41.612896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '952db8328a5e4d40849f62aefde55b88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:41.620031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '952db8328a5e4d40849f62aefde55b88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:41.620644 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '952db8328a5e4d40849f62aefde55b88'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:41.638185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1381/2768] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:41 2022] DELETE /identity/v3/users/952db8328a5e4d40849f62aefde55b88 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:41.642609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:41.643154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:41.643154 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:41.664109 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:41.666173 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:41.724225 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1388/2769] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:41.734509 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:41.736044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:41.773027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:41.797454 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c92157e-4837-4acc-a6b8-7ece6179ad83 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:41.801211 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:41.801824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:41.802137 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:41.802415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:41.811785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:41.813154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:41.823245 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1382/2770] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:42.093501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:42.094934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:42.108343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27cc7f-c07e-40b8-bd99-1d8046d71c8a None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:42.113285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:42.113954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.114266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.114541 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] PATH_INFO: `/v3/projects/1c8b67a4632b4f77886909f3d585c231` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.115666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c8b67a4632b4f77886909f3d585c231)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:42.117784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:42.164941 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c8b67a4632b4f77886909f3d585c231', 'request_id': 'req-2106646f-e638-40b7-a320-3bc26a240f94'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.170691 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c8b67a4632b4f77886909f3d585c231', 'request_id': 'req-2106646f-e638-40b7-a320-3bc26a240f94'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.178024 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c8b67a4632b4f77886909f3d585c231', 'request_id': 'req-2106646f-e638-40b7-a320-3bc26a240f94'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.189516 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] The token cache is being invalidate because project 1c8b67a4632b4f77886909f3d585c231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:42.189803 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.190482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.191937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1389/2771] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:42 2022] DELETE /identity/v3/projects/1c8b67a4632b4f77886909f3d585c231 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:42.196575 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:42.206122 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3368efc-926b-43e1-beef-8130cd936ecd service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:42.261041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:42.261667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.261946 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.262258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] PATH_INFO: `/v3/users/6f3b2518c0f24524b572d80e1cd0452e` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.263369 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f3b2518c0f24524b572d80e1cd0452e)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:42.265373 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:42.309550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f3b2518c0f24524b572d80e1cd0452e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.316838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f3b2518c0f24524b572d80e1cd0452e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.317472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f3b2518c0f24524b572d80e1cd0452e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.325212 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f3b2518c0f24524b572d80e1cd0452e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.325868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a1a4fe5a-1001-4e5e-8ef7-46fb3678ad2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f3b2518c0f24524b572d80e1cd0452e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.343001 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1383/2772] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:42 2022] DELETE /identity/v3/users/6f3b2518c0f24524b572d80e1cd0452e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:42.612203 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:42.613640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:42.627580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106646f-e638-40b7-a320-3bc26a240f94 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:42.658871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:42.691180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:42.691801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.692073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.692349 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] PATH_INFO: `/v3/projects/7b4cb5b4be7f48eaae4b7b1e7927eafb` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.693438 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4cb5b4be7f48eaae4b7b1e7927eafb)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:42.695494 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:42.739948 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4cb5b4be7f48eaae4b7b1e7927eafb', 'request_id': 'req-2a977440-d243-4ccb-bafc-50e76cd5e9af'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.745855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4cb5b4be7f48eaae4b7b1e7927eafb', 'request_id': 'req-2a977440-d243-4ccb-bafc-50e76cd5e9af'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.753237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4cb5b4be7f48eaae4b7b1e7927eafb', 'request_id': 'req-2a977440-d243-4ccb-bafc-50e76cd5e9af'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.764240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] The token cache is being invalidate because project 7b4cb5b4be7f48eaae4b7b1e7927eafb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:42.764525 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.765159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2a977440-d243-4ccb-bafc-50e76cd5e9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:42.766659 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1390/2773] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:42 2022] DELETE /identity/v3/projects/7b4cb5b4be7f48eaae4b7b1e7927eafb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:42.771467 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.771967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.772082 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.791838 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:42.793043 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:42.843051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:42.859173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1384/2774] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:42.866443 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.866731 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.867011 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.888437 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:42.889822 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:42.897818 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1391/2775] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:42.902670 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:42.913491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fca6f-5646-4089-ae47-d1ba4b4a8384 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:42.918766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:42.919366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.919745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.920020 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.920662 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:42.920957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:42.922576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:42.938034 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1385/2776] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:42.939474 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:42.948663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068cb484-7ddd-491b-a5c9-a5c27a3c34f0 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:42.953593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:42.954233 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:42.954513 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:42.954784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:42.955457 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:42.957348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:42.985905 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1392/2777] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:42 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:42.990363 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.002025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720116-d1a8-42d1-b6b2-1334ea2729f6 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.064423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.065044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.065320 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.065590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.066252 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.068184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.092487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1386/2778] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:42 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:43.096413 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.107911 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a07b-ccfc-4fbd-8119-0fec7acda157 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.112744 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.113477 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.113713 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.113980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.114648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.116224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.129672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1393/2779] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:43.131103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.141199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379f8f0-de9c-4d83-bc42-84f4fdbd934a admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.145691 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76458-e8c0-4c4c-95e9-ab6b07c3ba0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.146324 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76458-e8c0-4c4c-95e9-ab6b07c3ba0c admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.146601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76458-e8c0-4c4c-95e9-ab6b07c3ba0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.146872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76458-e8c0-4c4c-95e9-ab6b07c3ba0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.147499 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b76458-e8c0-4c4c-95e9-ab6b07c3ba0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.149642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b76458-e8c0-4c4c-95e9-ab6b07c3ba0c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.161099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1387/2780] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:43.165849 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.175998 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818456fa-cc5c-4547-8cba-304ad55af836 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.180675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.181316 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.181596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.181869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] PATH_INFO: `/v3/projects/f40f178df3c44aafae55f8831fac4f92/users/a8d689568c4446c4945b1dd32d92cd22/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.196550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40f178df3c44aafae55f8831fac4f92, user_id=a8d689568c4446c4945b1dd32d92cd22, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.199273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.212765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1394/2781] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:43 2022] PUT /identity/v3/projects/f40f178df3c44aafae55f8831fac4f92/users/a8d689568c4446c4945b1dd32d92cd22/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:43.223272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.223704 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.223991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.275458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] MFA Rules not processed for user `a8d689568c4446c4945b1dd32d92cd22`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:43.276667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:43.346972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1388/2782] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:43.351633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.409319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:43.415667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5b8972-d7ec-455c-95e8-409be65e2bb0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.421783 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.422435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.422715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.422986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.423627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.425550 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.452262 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1395/2783] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:43 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:43.456367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.467545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb02326-cc7a-4766-826f-7ebefd991239 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.516262 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:43.527749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.528352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.528626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.528890 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.529504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.531708 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.551654 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1389/2784] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:43 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:43.555763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.565718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01543f8c-41c3-42dd-8334-7d364685c5d3 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.570267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.571347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.571618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.571882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.572506 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.574107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.584219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1396/2785] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:43.588810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.598632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db00055-ea7e-42d2-9b71-4b5636594a31 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.603282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.603943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.604221 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.604495 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] PATH_INFO: `/v3/projects/826874385efc4086acd71dbd45ecef8b/users/65279ee9e7fd456fa3b83b9dff4e89b1/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.619015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=826874385efc4086acd71dbd45ecef8b, user_id=65279ee9e7fd456fa3b83b9dff4e89b1, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.621590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.634476 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1390/2786] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:43 2022] PUT /identity/v3/projects/826874385efc4086acd71dbd45ecef8b/users/65279ee9e7fd456fa3b83b9dff4e89b1/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:43.638369 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.648153 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9f333-42af-4700-8a8c-ea3d340cbfb0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.706297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da53f2-30dd-4bb2-9c9b-15172ed805c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.706999 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da53f2-30dd-4bb2-9c9b-15172ed805c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.707170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da53f2-30dd-4bb2-9c9b-15172ed805c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.707445 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da53f2-30dd-4bb2-9c9b-15172ed805c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.708075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da53f2-30dd-4bb2-9c9b-15172ed805c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.709658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da53f2-30dd-4bb2-9c9b-15172ed805c6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.719004 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1397/2787] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:43.723522 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.733547 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41e4c-2595-4b81-b2ad-6ac4f60535bc admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.738102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.738745 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.739021 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.739288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] PATH_INFO: `/v3/projects/826874385efc4086acd71dbd45ecef8b/users/65279ee9e7fd456fa3b83b9dff4e89b1/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.741158 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=826874385efc4086acd71dbd45ecef8b, user_id=65279ee9e7fd456fa3b83b9dff4e89b1, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.744307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.754243 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1391/2788] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:43 2022] PUT /identity/v3/projects/826874385efc4086acd71dbd45ecef8b/users/65279ee9e7fd456fa3b83b9dff4e89b1/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:43.765801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.766120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.766396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.793246 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] MFA Rules not processed for user `65279ee9e7fd456fa3b83b9dff4e89b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:43.794468 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:43.859613 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1398/2789] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:43.871197 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:43.872867 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:43.928770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334b59e-6ba6-4d30-bece-fa225f548b9d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:43.932880 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eeab4b-332e-4805-a7dc-eb8f0a674554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:43.933472 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeab4b-332e-4805-a7dc-eb8f0a674554 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:43.933749 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeab4b-332e-4805-a7dc-eb8f0a674554 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:43.934048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eeab4b-332e-4805-a7dc-eb8f0a674554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:43.942816 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeab4b-332e-4805-a7dc-eb8f0a674554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:43.944225 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eeab4b-332e-4805-a7dc-eb8f0a674554 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:43.954108 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1392/2790] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:43.982335 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:44.032219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:44.044933 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9092160-9474-482f-b241-8c72da6bc4cc None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:44.049520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:44.050183 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:44.050460 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:44.050728 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] PATH_INFO: `/v3/users/a8d689568c4446c4945b1dd32d92cd22` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:44.052180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d689568c4446c4945b1dd32d92cd22)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:44.054473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:44.099653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d689568c4446c4945b1dd32d92cd22'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.107305 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d689568c4446c4945b1dd32d92cd22'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.108025 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d689568c4446c4945b1dd32d92cd22'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.108654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d689568c4446c4945b1dd32d92cd22'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.115633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d689568c4446c4945b1dd32d92cd22'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.134434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1399/2791] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:43 2022] DELETE /identity/v3/users/a8d689568c4446c4945b1dd32d92cd22 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:44.138980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:44.139260 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:44.139531 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:44.158824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:44.160015 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:44.216706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1393/2792] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:44.226392 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:44.228265 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:44.284927 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea0c68-fa28-4811-9b78-addc14a4d280 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:44.288492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:44.289085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:44.289856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:44.290171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:44.298828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:44.300224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:44.310093 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1400/2793] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:44.574780 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:44.576615 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:44.590580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd707c-d143-4e33-b812-f13da19c7561 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:44.604518 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:44.605181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:44.605460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:44.605732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] PATH_INFO: `/v3/projects/f40f178df3c44aafae55f8831fac4f92` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:44.607054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40f178df3c44aafae55f8831fac4f92)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:44.609279 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:44.652828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:44.661754 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40f178df3c44aafae55f8831fac4f92', 'request_id': 'req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.672124 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40f178df3c44aafae55f8831fac4f92', 'request_id': 'req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.678959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40f178df3c44aafae55f8831fac4f92', 'request_id': 'req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.690270 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] The token cache is being invalidate because project f40f178df3c44aafae55f8831fac4f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:44.690632 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.691295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.692778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1394/2794] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:44 2022] DELETE /identity/v3/projects/f40f178df3c44aafae55f8831fac4f92 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:44.697003 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:44.706575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d931e9b-bfc5-4fbc-88dd-ea7f9e6cd943 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:44.763633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:44.764264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:44.764544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:44.764814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] PATH_INFO: `/v3/users/65279ee9e7fd456fa3b83b9dff4e89b1` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:44.765923 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65279ee9e7fd456fa3b83b9dff4e89b1)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:44.767871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:44.781809 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:44.812914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65279ee9e7fd456fa3b83b9dff4e89b1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.819956 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65279ee9e7fd456fa3b83b9dff4e89b1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.821158 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65279ee9e7fd456fa3b83b9dff4e89b1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.821754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65279ee9e7fd456fa3b83b9dff4e89b1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.828557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-2d5a385b-31f1-498e-ba15-ef827dcca784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65279ee9e7fd456fa3b83b9dff4e89b1'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:44.845507 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1401/2795] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:44 2022] DELETE /identity/v3/users/65279ee9e7fd456fa3b83b9dff4e89b1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:45.121040 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.123060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:45.149628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba4e9b-b4aa-4742-8d4f-f0438b02e84e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.210352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.210998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.211330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.211607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] PATH_INFO: `/v3/projects/826874385efc4086acd71dbd45ecef8b` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.212744 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=826874385efc4086acd71dbd45ecef8b)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.214718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.276384 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826874385efc4086acd71dbd45ecef8b', 'request_id': 'req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:45.284217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826874385efc4086acd71dbd45ecef8b', 'request_id': 'req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:45.290712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826874385efc4086acd71dbd45ecef8b', 'request_id': 'req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:45.302120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] The token cache is being invalidate because project 826874385efc4086acd71dbd45ecef8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:45.302411 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:45.303048 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-a29892d9-3ee8-4652-bdf1-fa9f589a8f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:45.304521 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1395/2796] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:45 2022] DELETE /identity/v3/projects/826874385efc4086acd71dbd45ecef8b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:45.309967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.310342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.310678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.332131 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:45.333329 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:45.393022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1402/2797] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:45.400519 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.400967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.401405 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.423343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:45.424576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:45.433844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1396/2798] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:45.437901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.448400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a072c-c263-4153-8a51-a26f40a4ce6c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.460073 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.460780 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.461040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.461322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.462606 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:45.462914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.464578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.479662 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1403/2799] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:45.483705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.494542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec2640-cd56-4418-a0c0-e9671f4bc5e0 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.499357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.499996 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.500271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.500542 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.501840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.503360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.543256 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1397/2800] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:45.547448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.559185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98213eea-f45b-49be-b1e1-22fdf0245392 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.624737 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.625466 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.625835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.626195 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.626840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.628724 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.653152 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1404/2801] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:45.658325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.670146 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275054e-8d0f-4a6d-ad3f-db385bd67a47 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.674823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.675424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.675699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.675976 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.676604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.678207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.693487 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1398/2802] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:45.694324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.704939 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d5d37-98fb-4354-812f-f59a5b3e8424 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.711644 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2ff522-3ce7-4d34-8623-cf44ddd7f29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.712626 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ff522-3ce7-4d34-8623-cf44ddd7f29c admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.712908 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ff522-3ce7-4d34-8623-cf44ddd7f29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.713185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2ff522-3ce7-4d34-8623-cf44ddd7f29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.713837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ff522-3ce7-4d34-8623-cf44ddd7f29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.716440 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2ff522-3ce7-4d34-8623-cf44ddd7f29c admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.718931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:45.730038 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1405/2803] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:45.733929 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.745254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb734c6-7d5f-4f4d-abe9-7dde05c6e176 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.750517 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.751176 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.751452 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.751721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] PATH_INFO: `/v3/projects/75a1a253881e46e68ad2f022be809368/users/71c3bb1474cd4a758cbd6a2f282d88c3/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.768551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a1a253881e46e68ad2f022be809368, user_id=71c3bb1474cd4a758cbd6a2f282d88c3, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.771250 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.774491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:45.786377 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1399/2804] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:45 2022] PUT /identity/v3/projects/75a1a253881e46e68ad2f022be809368/users/71c3bb1474cd4a758cbd6a2f282d88c3/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:45.797117 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.797401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.797668 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.824804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] MFA Rules not processed for user `71c3bb1474cd4a758cbd6a2f282d88c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:45.826015 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:45.888273 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1406/2805] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:45.892831 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:45.954949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941f6a9-fc8a-466a-9f69-4119c93e037d admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:45.959594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:45.960193 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:45.960478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:45.960750 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:45.962016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:45.963859 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:45.991086 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1400/2806] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:45.995133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.005945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962658af-d7c8-4670-8521-f98ecf91decb None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.071076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.071775 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.072051 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.072324 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.072951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.074910 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.097832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1407/2807] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:45 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:46.101602 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.113549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6b993-e4f3-4c3d-a9ff-596b8269f010 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.118345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.118967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.119246 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.119516 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.120186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.121785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.133009 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1401/2808] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:46.137902 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.150675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488d118-1e66-4d4a-9cfe-4aa24e5a4f31 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.155558 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.156207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.156491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.156766 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] PATH_INFO: `/v3/projects/98fc3ef57e5941a99794bdadd8421f6b/users/a18590977ecd4fe6bfc907c2ae96a6ca/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.172035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fc3ef57e5941a99794bdadd8421f6b, user_id=a18590977ecd4fe6bfc907c2ae96a6ca, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.174815 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.187449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1408/2809] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:46 2022] PUT /identity/v3/projects/98fc3ef57e5941a99794bdadd8421f6b/users/a18590977ecd4fe6bfc907c2ae96a6ca/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:46.191367 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.201307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d13db3-f894-4f1f-8fbb-3f1ce4013f16 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.256337 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25dc0e-ebed-4b70-a069-652f838ade0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.256872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25dc0e-ebed-4b70-a069-652f838ade0f admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.257149 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25dc0e-ebed-4b70-a069-652f838ade0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.257419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25dc0e-ebed-4b70-a069-652f838ade0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.258581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25dc0e-ebed-4b70-a069-652f838ade0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.260160 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25dc0e-ebed-4b70-a069-652f838ade0f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.269712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1402/2810] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:46.274730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.285063 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acc9f2-6638-4bcc-abad-ef541897bcf6 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.289585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.290258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.290535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.290806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] PATH_INFO: `/v3/projects/98fc3ef57e5941a99794bdadd8421f6b/users/a18590977ecd4fe6bfc907c2ae96a6ca/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.292697 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fc3ef57e5941a99794bdadd8421f6b, user_id=a18590977ecd4fe6bfc907c2ae96a6ca, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.295358 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.305264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1409/2811] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:46 2022] PUT /identity/v3/projects/98fc3ef57e5941a99794bdadd8421f6b/users/a18590977ecd4fe6bfc907c2ae96a6ca/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:46.316399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.316687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.316960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.345325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] MFA Rules not processed for user `a18590977ecd4fe6bfc907c2ae96a6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:46.346788 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:46.397254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:46.416628 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1403/2812] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:46.427708 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.429408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:46.474408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:46.488848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21dc7c-8e88-45a6-a49b-420189725d67 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.492508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137fbab-cd6f-4e90-b95f-900959c3f807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.493103 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137fbab-cd6f-4e90-b95f-900959c3f807 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.493375 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137fbab-cd6f-4e90-b95f-900959c3f807 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.493640 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137fbab-cd6f-4e90-b95f-900959c3f807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.502486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137fbab-cd6f-4e90-b95f-900959c3f807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.503882 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137fbab-cd6f-4e90-b95f-900959c3f807 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.513623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1410/2813] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:46.579410 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.640752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e813be-ef40-4a0a-bccc-d0ae7800d844 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.645281 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.645907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.646214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.646485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] PATH_INFO: `/v3/users/71c3bb1474cd4a758cbd6a2f282d88c3` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.647612 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c3bb1474cd4a758cbd6a2f282d88c3)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.650186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.699938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c3bb1474cd4a758cbd6a2f282d88c3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:46.708092 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c3bb1474cd4a758cbd6a2f282d88c3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:46.708714 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c3bb1474cd4a758cbd6a2f282d88c3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:46.715977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c3bb1474cd4a758cbd6a2f282d88c3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:46.716599 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c3bb1474cd4a758cbd6a2f282d88c3'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:46.733408 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1404/2814] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:46 2022] DELETE /identity/v3/users/71c3bb1474cd4a758cbd6a2f282d88c3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:46.737894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511041bb-351b-4428-af89-b1adde48a6db None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.738435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511041bb-351b-4428-af89-b1adde48a6db None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.738435 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511041bb-351b-4428-af89-b1adde48a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.757244 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-511041bb-351b-4428-af89-b1adde48a6db None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:46.759104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-511041bb-351b-4428-af89-b1adde48a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:46.821352 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1411/2815] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:46.829858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:46.831288 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:46.891401 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a26a66-9a2f-4720-bbfa-8a1986237f91 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:46.898415 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:46.899220 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.899441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.899635 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.909214 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:46.910876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:46.922177 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1405/2816] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:46.955429 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:46.955714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:46.955983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:46.976449 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:46.977717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:47.036808 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1412/2817] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:14:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:47.049339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:47.050846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:47.070690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:47.127469 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b443ac7-9bfa-415e-851c-db4ceed45773 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:47.132828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:47.133577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.133860 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.134173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.142568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:47.144151 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:47.152244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1406/2818] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:14:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 05:14:47.207593 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:47.209058 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:47.226214 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f61109-6568-4c2d-aeba-ec9e306717c9 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:47.232592 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:47.233469 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.233755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.234135 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] PATH_INFO: `/v3/projects/75a1a253881e46e68ad2f022be809368` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.235370 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a1a253881e46e68ad2f022be809368)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:47.237499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:47.293716 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a1a253881e46e68ad2f022be809368', 'request_id': 'req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.302831 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a1a253881e46e68ad2f022be809368', 'request_id': 'req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.310723 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a1a253881e46e68ad2f022be809368', 'request_id': 'req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.323020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] The token cache is being invalidate because project 75a1a253881e46e68ad2f022be809368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:47.323314 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.323969 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-cd1e8e72-7b6a-4413-a2dd-7f24480a4bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.325458 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1413/2819] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:47 2022] DELETE /identity/v3/projects/75a1a253881e46e68ad2f022be809368 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:47.329589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:47.340152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770db3b-c994-4a40-9b6b-7c967dcce4e8 service swift] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:47.409882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:47.410554 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.410833 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.411110 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] PATH_INFO: `/v3/users/a18590977ecd4fe6bfc907c2ae96a6ca` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.412247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a18590977ecd4fe6bfc907c2ae96a6ca)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:47.414752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:47.464106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a18590977ecd4fe6bfc907c2ae96a6ca'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.471392 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a18590977ecd4fe6bfc907c2ae96a6ca'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.472293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a18590977ecd4fe6bfc907c2ae96a6ca'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.482240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a18590977ecd4fe6bfc907c2ae96a6ca'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.482936 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a18590977ecd4fe6bfc907c2ae96a6ca'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.500639 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1407/2820] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:47 2022] DELETE /identity/v3/users/a18590977ecd4fe6bfc907c2ae96a6ca => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:47.714253 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.714544 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.714824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.735201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:47.736421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:47.742885 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:47.790576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:47.792482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:47.800853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1414/2821] 10.222.0.50 () {58 vars in 967 bytes} [Sat Sep 24 05:14:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:47.814953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:47.816401 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:47.825533 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:47.866972 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24a318-55c3-4bc6-b9da-d3ad8d3d8b3f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:47.871756 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:47.872465 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.872815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.873153 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] PATH_INFO: `/v3/projects/98fc3ef57e5941a99794bdadd8421f6b` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.874442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98fc3ef57e5941a99794bdadd8421f6b)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:47.877115 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:47.889292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3c146-f4aa-457c-996e-e7e682007815 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:47.894729 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ad765-d2bf-4b03-98ac-c3f6c7abde1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:47.895343 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad765-d2bf-4b03-98ac-c3f6c7abde1f service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.895623 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad765-d2bf-4b03-98ac-c3f6c7abde1f service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.895894 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ad765-d2bf-4b03-98ac-c3f6c7abde1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.904648 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ad765-d2bf-4b03-98ac-c3f6c7abde1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:47.906907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ad765-d2bf-4b03-98ac-c3f6c7abde1f service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:47.916486 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1415/2822] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:14:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 05:14:47.927844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98fc3ef57e5941a99794bdadd8421f6b', 'request_id': 'req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.935224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98fc3ef57e5941a99794bdadd8421f6b', 'request_id': 'req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.944258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98fc3ef57e5941a99794bdadd8421f6b', 'request_id': 'req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.956275 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] The token cache is being invalidate because project 98fc3ef57e5941a99794bdadd8421f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:47.956577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.957368 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-c9aeb953-d96c-4fc2-b99b-5ef7ad36d130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:47.958896 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1408/2823] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:47 2022] DELETE /identity/v3/projects/98fc3ef57e5941a99794bdadd8421f6b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:47.964377 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:47.964663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:47.964943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:47.985473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:47.986765 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:48.057193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1416/2824] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:48.065736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.066056 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.066342 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.094725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:48.094725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:48.099737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1409/2825] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:48.103895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.113404 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ee8ca-4452-40ef-bf4c-0f1a3455a2b7 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.118086 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.118686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.118967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.119357 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.119875 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:48.120169 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.121748 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e27415-afec-406c-b780-137c69be389a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.135284 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1417/2826] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:48.139706 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.149834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b35f41-e47c-4f2f-b807-bcce9fcc83d3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.157812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.158237 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.158484 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.158620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.158765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.158877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.159027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.159667 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.161653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.180442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:48.181670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:48.189282 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1410/2827] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:48.193483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.212679 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:48.240252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1418/2828] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:48.253035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.254492 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:48.256642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f662433-c1de-4b26-82f7-3a9ae02d6bfb admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.261365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.261987 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.262290 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.262562 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.263186 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.265080 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.295638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1411/2829] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:48.296876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.306743 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1bc11-1a98-430c-9d4d-66f6b8387351 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.311491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.312022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.312301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.312577 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.313205 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.314865 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.325239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1412/2830] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:48.328981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.332348 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d122601c-faf8-4f98-992a-9fe688adce6b None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.336530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:48.344995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b1829-44da-4c55-8bbd-0d6916293776 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.350594 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.351203 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.351772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.351772 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.352947 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.354581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.366060 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1413/2831] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:48.370170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.380219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133cc31-9c11-4357-a6f8-ec6e3966e7b1 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.384830 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284a5c4-7f86-43ec-934c-650f5a450b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.385460 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284a5c4-7f86-43ec-934c-650f5a450b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.385732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284a5c4-7f86-43ec-934c-650f5a450b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.386027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284a5c4-7f86-43ec-934c-650f5a450b8c admin admin] PATH_INFO: `/v3/projects/cea84f4921bd450c9b66669d78396ef7/users/bd0f4b6677e04825b6c396de8463f338/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.400802 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284a5c4-7f86-43ec-934c-650f5a450b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea84f4921bd450c9b66669d78396ef7, user_id=bd0f4b6677e04825b6c396de8463f338, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.403266 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.403482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284a5c4-7f86-43ec-934c-650f5a450b8c admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.403865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.404137 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.404412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.418572 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1414/2832] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:48 2022] PUT /identity/v3/projects/cea84f4921bd450c9b66669d78396ef7/users/bd0f4b6677e04825b6c396de8463f338/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:48.428886 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.429175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.429459 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.456783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] MFA Rules not processed for user `bd0f4b6677e04825b6c396de8463f338`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:48.458148 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:48.467726 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.469302 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.479603 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1419/2833] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 05:14:48.526209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1415/2834] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:48.530749 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.542171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3676cc6-6402-4980-a48c-1c23c7785da7 service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.560692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:48.602317 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.603005 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.603342 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.603678 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.604517 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.606409 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.634854 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1420/2835] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:48.638625 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.653159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e20e4-a63a-4897-b09d-6d4f117b12ee None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.715681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.716298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.716579 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.716863 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.717483 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.719425 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.742035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1416/2836] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:48.745987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.758586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e994ed-1ad0-4478-a1ef-80528226fbcc admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.763682 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a53f6-d59a-4553-8d04-895686d68c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.764277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a53f6-d59a-4553-8d04-895686d68c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.764594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a53f6-d59a-4553-8d04-895686d68c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.764821 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a53f6-d59a-4553-8d04-895686d68c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.765434 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a53f6-d59a-4553-8d04-895686d68c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.767033 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a53f6-d59a-4553-8d04-895686d68c5b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.778532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1421/2837] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:48.782960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.793239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f45ff-3210-4b55-867f-153977ef6630 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.798273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.798944 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.799226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.799506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] PATH_INFO: `/v3/projects/3c9cf54560914896bf8990dfb97ab5e9/users/1960ec3f3bac4f38bd841043c6c536b7/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.815204 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9cf54560914896bf8990dfb97ab5e9, user_id=1960ec3f3bac4f38bd841043c6c536b7, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.817512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.817858 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.817967 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.818308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.833606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1417/2838] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:48 2022] PUT /identity/v3/projects/3c9cf54560914896bf8990dfb97ab5e9/users/1960ec3f3bac4f38bd841043c6c536b7/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:48.838025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.839159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:48.840347 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:48.862821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:48.897505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1422/2839] 10.222.0.50 () {58 vars in 967 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:48.900854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc63258-3ae5-44ff-8382-4e11384ab62f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.905829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.906471 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.906751 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.907023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.907657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.909248 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.910408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.922847 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1418/2840] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:48.924139 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:48.934105 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ad5b7-3809-485a-8cec-73718952ea15 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.938643 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648c2d4-0972-4ee1-9fcf-b8d1e2aadad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:48.939263 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648c2d4-0972-4ee1-9fcf-b8d1e2aadad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.939539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648c2d4-0972-4ee1-9fcf-b8d1e2aadad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.939811 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648c2d4-0972-4ee1-9fcf-b8d1e2aadad1 admin admin] PATH_INFO: `/v3/projects/3c9cf54560914896bf8990dfb97ab5e9/users/1960ec3f3bac4f38bd841043c6c536b7/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.941713 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648c2d4-0972-4ee1-9fcf-b8d1e2aadad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9cf54560914896bf8990dfb97ab5e9, user_id=1960ec3f3bac4f38bd841043c6c536b7, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:48.944364 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648c2d4-0972-4ee1-9fcf-b8d1e2aadad1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:48.957213 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1419/2841] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:48 2022] PUT /identity/v3/projects/3c9cf54560914896bf8990dfb97ab5e9/users/1960ec3f3bac4f38bd841043c6c536b7/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:48.968366 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:48.968874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:48.969145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:48.978280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28ab5b-c259-4708-a4d4-25446b8443de None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:48.997033 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] MFA Rules not processed for user `1960ec3f3bac4f38bd841043c6c536b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:48.998271 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:49.020459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:49.040757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:49.041372 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:49.041650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:49.041919 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:49.069321 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1420/2842] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:14:49.080420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:49.081797 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:49.099119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:49.100642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:49.109752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1423/2843] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:14:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 24 05:14:49.151769 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57228387-6ad2-440d-a6da-2c1bbfe2bc55 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:49.155923 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a338b-1bdf-4da0-96ba-ad609e775229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:49.156543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a338b-1bdf-4da0-96ba-ad609e775229 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:49.156824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a338b-1bdf-4da0-96ba-ad609e775229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:49.157102 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a338b-1bdf-4da0-96ba-ad609e775229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:49.167737 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a338b-1bdf-4da0-96ba-ad609e775229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:49.169444 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a338b-1bdf-4da0-96ba-ad609e775229 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:49.179878 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1421/2844] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:49 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:49.569826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:49.636743 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2dd0d-d65e-42a6-9216-b983fe684af1 service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:49.641547 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:49.642297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:49.642649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:49.642980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] PATH_INFO: `/v3/users/bd0f4b6677e04825b6c396de8463f338` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:49.644866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0f4b6677e04825b6c396de8463f338)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:49.651210 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:49.700079 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0f4b6677e04825b6c396de8463f338'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:49.708331 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0f4b6677e04825b6c396de8463f338'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:49.709105 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0f4b6677e04825b6c396de8463f338'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:49.709901 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0f4b6677e04825b6c396de8463f338'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:49.718937 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0f4b6677e04825b6c396de8463f338'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:49.739258 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1424/2845] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:49 2022] DELETE /identity/v3/users/bd0f4b6677e04825b6c396de8463f338 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:49.744458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:49.744820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:49.745237 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:49.766546 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:49.767834 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:49.799167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:49.833977 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1422/2846] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:49.844092 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:49.845596 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:49.852108 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:49.906077 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c315dd9-89ee-45f9-b0dd-e51441d22910 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:49.911667 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:49.912267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:49.912534 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:49.912799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:49.921959 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:49.923400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:49.934855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1425/2847] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:50.224409 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:50.226199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:50.244437 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc668f-f4cd-4320-b12b-04709a978032 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:50.249334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:50.250025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:50.250330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:50.250604 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] PATH_INFO: `/v3/projects/cea84f4921bd450c9b66669d78396ef7` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:50.251755 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea84f4921bd450c9b66669d78396ef7)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:50.253687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:50.305181 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea84f4921bd450c9b66669d78396ef7', 'request_id': 'req-721f51d3-6fa0-439c-b663-4ed281b79d95'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.312804 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea84f4921bd450c9b66669d78396ef7', 'request_id': 'req-721f51d3-6fa0-439c-b663-4ed281b79d95'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.319586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea84f4921bd450c9b66669d78396ef7', 'request_id': 'req-721f51d3-6fa0-439c-b663-4ed281b79d95'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.331507 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] The token cache is being invalidate because project cea84f4921bd450c9b66669d78396ef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:50.331782 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.332420 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.334289 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1423/2848] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:50 2022] DELETE /identity/v3/projects/cea84f4921bd450c9b66669d78396ef7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:50.338288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:50.348423 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df273c-8880-4187-98cd-9f094a337d28 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:50.407097 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:50.407647 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:50.407924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:50.408193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] PATH_INFO: `/v3/users/1960ec3f3bac4f38bd841043c6c536b7` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:50.409319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1960ec3f3bac4f38bd841043c6c536b7)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:50.411295 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:50.455631 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1960ec3f3bac4f38bd841043c6c536b7'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.463605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1960ec3f3bac4f38bd841043c6c536b7'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.464224 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1960ec3f3bac4f38bd841043c6c536b7'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.464853 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1960ec3f3bac4f38bd841043c6c536b7'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.471983 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-05ba3ac7-9b5a-42f2-ab1a-30c1bba9d939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1960ec3f3bac4f38bd841043c6c536b7'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.490341 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1426/2849] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:50 2022] DELETE /identity/v3/users/1960ec3f3bac4f38bd841043c6c536b7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:50.772357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:50.774435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:50.788168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f51d3-6fa0-439c-b663-4ed281b79d95 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:50.846042 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:50.846677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:50.846955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:50.847228 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] PATH_INFO: `/v3/projects/3c9cf54560914896bf8990dfb97ab5e9` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:50.848340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9cf54560914896bf8990dfb97ab5e9)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:50.850274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:50.894497 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9cf54560914896bf8990dfb97ab5e9', 'request_id': 'req-877d0a2a-aabe-450d-9a67-7d226a388f75'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.902637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9cf54560914896bf8990dfb97ab5e9', 'request_id': 'req-877d0a2a-aabe-450d-9a67-7d226a388f75'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.904679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:50.909684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9cf54560914896bf8990dfb97ab5e9', 'request_id': 'req-877d0a2a-aabe-450d-9a67-7d226a388f75'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.921963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] The token cache is being invalidate because project 3c9cf54560914896bf8990dfb97ab5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:50.922269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.922895 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-877d0a2a-aabe-450d-9a67-7d226a388f75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:50.924343 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1424/2850] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:50 2022] DELETE /identity/v3/projects/3c9cf54560914896bf8990dfb97ab5e9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:50.929112 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:50.929565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:50.929714 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:50.949421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:50.950595 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:50.970044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:51.009750 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1427/2851] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:51.017589 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.017871 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.018187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.037539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:51.038765 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.049568 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1425/2852] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:51.053712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.064180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc43c4-cc0a-4e5f-8f7d-8458a329641c None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.071171 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.071772 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.072642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.072642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.072989 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:51.073291 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.074980 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.086801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1428/2853] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:14:51.091167 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.100224 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8634b14-4eec-4344-b88b-935a74c8a145 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.105259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.105846 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.106159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.106433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.107052 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.108957 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.140109 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1426/2854] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:51 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:51.141263 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.151559 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72e61b-8061-4c66-9356-947959d44a07 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.208315 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.208920 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.209200 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.209473 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.210138 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.212574 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.233380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1429/2855] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:51 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:14:51.237709 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.247881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e4302-744d-436f-9a4d-ef95a3f955d1 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.252433 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.253026 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.253307 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.253576 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.254346 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.255941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.265640 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1427/2856] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:51.269218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.281107 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902c1b3-35b9-4d81-bd38-ed577603d7e5 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.285650 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17704e59-d6eb-4fb6-a98e-5819a8c1fba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.286285 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17704e59-d6eb-4fb6-a98e-5819a8c1fba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.286880 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17704e59-d6eb-4fb6-a98e-5819a8c1fba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.287142 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17704e59-d6eb-4fb6-a98e-5819a8c1fba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.287776 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17704e59-d6eb-4fb6-a98e-5819a8c1fba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.289887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17704e59-d6eb-4fb6-a98e-5819a8c1fba9 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.300292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1430/2857] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:51.304530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.314375 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b66dbd-9f30-4208-a21a-f5a04b69de34 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.318963 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.319642 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.319959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.320273 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] PATH_INFO: `/v3/projects/e6c1e44c9b4b439dabfc60cf9771b38f/users/1c55b12e5587414983cc4c2925d68284/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.334679 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c1e44c9b4b439dabfc60cf9771b38f, user_id=1c55b12e5587414983cc4c2925d68284, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.337828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.351606 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1428/2858] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:51 2022] PUT /identity/v3/projects/e6c1e44c9b4b439dabfc60cf9771b38f/users/1c55b12e5587414983cc4c2925d68284/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:51.363277 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.363560 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.363832 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.390627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] MFA Rules not processed for user `1c55b12e5587414983cc4c2925d68284`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:51.391828 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.426948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.428770 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.457218 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1431/2859] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:51.469535 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.470950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.494352 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9ca33-5275-42d2-9c13-82854c3a2182 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.498371 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:51.499145 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.499778 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.500061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.500334 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] PATH_INFO: `/v3/users/abf719b5d31043cc937217f4610e4190` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.501478 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abf719b5d31043cc937217f4610e4190)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.503419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.531987 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1920a6-ada3-4f53-bdd9-c16696fc14e2 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.535953 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae74581-1c0a-41de-a73e-3cc935d9ec54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9754c63d24741be93896d6ef83be3a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.536599 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae74581-1c0a-41de-a73e-3cc935d9ec54 service glance] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.536943 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae74581-1c0a-41de-a73e-3cc935d9ec54 service glance] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.537274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae74581-1c0a-41de-a73e-3cc935d9ec54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.545790 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae74581-1c0a-41de-a73e-3cc935d9ec54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.547252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae74581-1c0a-41de-a73e-3cc935d9ec54 service glance] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.550794 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abf719b5d31043cc937217f4610e4190'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:51.556298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1432/2860] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 24 05:14:51.558207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abf719b5d31043cc937217f4610e4190'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:51.558991 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abf719b5d31043cc937217f4610e4190'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:51.565984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abf719b5d31043cc937217f4610e4190'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:51.566657 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abf719b5d31043cc937217f4610e4190'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:51.590820 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1429/2861] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:14:51 2022] DELETE /identity/v3/users/abf719b5d31043cc937217f4610e4190 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:14:51.595123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.595416 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.595694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.614879 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:51.616075 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.681784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1433/2862] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:51.691075 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.692550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.698489 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:51.755553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd686dd-2fa6-4c42-9637-0ccb9acbd765 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.759868 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de4eb0-d42e-43ba-ad9b-b504a6a0030f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.760504 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de4eb0-d42e-43ba-ad9b-b504a6a0030f service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.760808 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de4eb0-d42e-43ba-ad9b-b504a6a0030f service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.761081 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de4eb0-d42e-43ba-ad9b-b504a6a0030f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.770485 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de4eb0-d42e-43ba-ad9b-b504a6a0030f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.771948 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de4eb0-d42e-43ba-ad9b-b504a6a0030f service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:51.782349 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1430/2863] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:14:51.958462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:51.960289 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:51.974580 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f04a0-e448-4714-988b-7299f5915492 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:51.986206 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:51.987219 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:51.987505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:51.987784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] PATH_INFO: `/v3/projects/3c7dd358b27c46c39025fc95f3d56997` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:51.989116 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7dd358b27c46c39025fc95f3d56997)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:51.991016 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.042476 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7dd358b27c46c39025fc95f3d56997', 'request_id': 'req-a0e96ea2-152b-4130-be4e-e2572c99a3f6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:52.048076 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7dd358b27c46c39025fc95f3d56997', 'request_id': 'req-a0e96ea2-152b-4130-be4e-e2572c99a3f6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:52.055568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7dd358b27c46c39025fc95f3d56997', 'request_id': 'req-a0e96ea2-152b-4130-be4e-e2572c99a3f6'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:52.066881 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] The token cache is being invalidate because project 3c7dd358b27c46c39025fc95f3d56997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:14:52.067165 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:52.067804 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-a0e96ea2-152b-4130-be4e-e2572c99a3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:14:52.069286 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1434/2864] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:14:51 2022] DELETE /identity/v3/projects/3c7dd358b27c46c39025fc95f3d56997 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:52.074626 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.074907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.075174 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.094357 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:52.095550 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:52.153786 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1431/2865] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:52.162702 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.162996 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.163272 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.185704 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:52.186995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:52.195019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1435/2866] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:52.199735 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.209274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2cc0b-f5e9-43e0-b515-67749c887ba3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.214111 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.214718 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.215022 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.215322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.216063 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:14:52.216305 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.217885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.229390 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1432/2867] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 24 05:14:52.233578 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.243191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04b5fc-96c8-481b-b0ac-e8f709b19719 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.247835 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.248424 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.248730 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.249008 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.249642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.251672 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.280061 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1436/2868] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:14:52 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:52.284400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.294951 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad633fa-da42-4b92-8f93-86e03991cea8 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.356166 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.356801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.357093 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.357362 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.358035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.360027 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.383077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1433/2869] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:14:52 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:14:52.387814 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.398470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75216c5d-d72a-45de-bbbc-fb37c40469d4 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.402995 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.403587 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.403865 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.404128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.404755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.406632 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.416801 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1437/2870] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 24 05:14:52.421103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.432162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137729c0-5b0b-4dda-93ae-5929b2eb06b9 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.436905 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c65846-5640-4e4a-81e1-4843c4eaceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.437503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c65846-5640-4e4a-81e1-4843c4eaceda admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.437784 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c65846-5640-4e4a-81e1-4843c4eaceda admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.438106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c65846-5640-4e4a-81e1-4843c4eaceda admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.438747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c65846-5640-4e4a-81e1-4843c4eaceda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.440350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c65846-5640-4e4a-81e1-4843c4eaceda admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.452191 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1434/2871] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:14:52.456346 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.466396 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427fea6-7138-40ac-be9b-3b4a3c50b21b admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.470922 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.471785 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.472087 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.472425 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] PATH_INFO: `/v3/projects/2c6a1aa9a5134f95bdd43edae061ac05/users/da15d6646b524fe7be500be2a1324489/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.476972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:52.490986 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6a1aa9a5134f95bdd43edae061ac05, user_id=da15d6646b524fe7be500be2a1324489, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.494508 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.509967 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1438/2872] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:14:52 2022] PUT /identity/v3/projects/2c6a1aa9a5134f95bdd43edae061ac05/users/da15d6646b524fe7be500be2a1324489/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:14:52.520821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.521118 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.521388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.543303 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:14:52.551825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] MFA Rules not processed for user `da15d6646b524fe7be500be2a1324489`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:14:52.553235 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:52.618423 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1435/2873] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:14:52 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:14:52.630170 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:14:52.631840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:14:52.690763 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb3429-2901-494e-a909-028733eeb139 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:14:52.694715 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bffff-410c-49b5-ac85-7656eb9ae861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:14:52.695318 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bffff-410c-49b5-ac85-7656eb9ae861 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:14:52.695604 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bffff-410c-49b5-ac85-7656eb9ae861 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:14:52.695877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bffff-410c-49b5-ac85-7656eb9ae861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:14:52.704111 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bffff-410c-49b5-ac85-7656eb9ae861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:14:52.705542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bffff-410c-49b5-ac85-7656eb9ae861 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:14:52.715220 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1439/2874] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:14:52 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:02.614943 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=27 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:02.710459 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:04.926556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:05.002069 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d3bbbb-1239-4d25-be17-4bf1ef0c4ce7 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:05.006686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:05.007322 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:05.007596 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:05.008041 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] PATH_INFO: `/v3/users/da15d6646b524fe7be500be2a1324489` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:05.009169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] RBAC: Authorizing `identity:delete_user(user_id=da15d6646b524fe7be500be2a1324489)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:05.011152 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:05.062244 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da15d6646b524fe7be500be2a1324489'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.070449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da15d6646b524fe7be500be2a1324489'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.071511 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da15d6646b524fe7be500be2a1324489'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.079885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da15d6646b524fe7be500be2a1324489'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.080835 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da15d6646b524fe7be500be2a1324489'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.104683 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1436/2875] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:15:04 2022] DELETE /identity/v3/users/da15d6646b524fe7be500be2a1324489 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:05.109199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:05.109712 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:05.109951 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:05.134947 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:05.136280 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:05.197007 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1440/2876] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:05.206558 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:05.207953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:05.267690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44793c7a-2ef6-4d91-8af7-17f529cc635f admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:05.272394 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:05.273036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:05.273313 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:05.273583 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:05.284120 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:05.285545 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:05.295442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1437/2877] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:05.561958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:05.563481 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:05.576946 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c26818-0537-4e35-ae8e-0e648687949f None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:05.581719 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:05.582470 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:05.582747 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:05.583670 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] PATH_INFO: `/v3/projects/2c6a1aa9a5134f95bdd43edae061ac05` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:05.585045 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6a1aa9a5134f95bdd43edae061ac05)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:05.587023 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:05.635261 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6a1aa9a5134f95bdd43edae061ac05', 'request_id': 'req-d417fa73-6a02-4a4c-b69c-57758b32b351'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.641162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6a1aa9a5134f95bdd43edae061ac05', 'request_id': 'req-d417fa73-6a02-4a4c-b69c-57758b32b351'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.649217 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6a1aa9a5134f95bdd43edae061ac05', 'request_id': 'req-d417fa73-6a02-4a4c-b69c-57758b32b351'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.661848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] The token cache is being invalidate because project 2c6a1aa9a5134f95bdd43edae061ac05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:15:05.662168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.662807 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-d417fa73-6a02-4a4c-b69c-57758b32b351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:05.664307 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1441/2878] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:15:05 2022] DELETE /identity/v3/projects/2c6a1aa9a5134f95bdd43edae061ac05 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:15.290690 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:15.658755 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:20.850254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:20.851688 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:20.864885 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beabd58-ebf3-4cc6-b2d4-e02646492e96 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:20.920844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:20.921399 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:20.921678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:20.921949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] PATH_INFO: `/v3/users/1c55b12e5587414983cc4c2925d68284` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:20.923240 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c55b12e5587414983cc4c2925d68284)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:20.925159 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:20.969207 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c55b12e5587414983cc4c2925d68284'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:20.976217 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c55b12e5587414983cc4c2925d68284'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:20.976854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c55b12e5587414983cc4c2925d68284'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:20.984400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c55b12e5587414983cc4c2925d68284'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:20.985025 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c55b12e5587414983cc4c2925d68284'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:21.002388 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1438/2879] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:15:20 2022] DELETE /identity/v3/users/1c55b12e5587414983cc4c2925d68284 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:21.006845 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.007338 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.007462 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.026532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:21.027742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:21.091311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1442/2880] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:21.101236 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.102698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:21.161350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede574e-b3e5-44e2-b89f-7a1b03e4727e admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.165051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cde51b-a14f-4713-86ed-a04c21e64892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.165656 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cde51b-a14f-4713-86ed-a04c21e64892 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.165937 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cde51b-a14f-4713-86ed-a04c21e64892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.166254 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cde51b-a14f-4713-86ed-a04c21e64892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.176046 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cde51b-a14f-4713-86ed-a04c21e64892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.177419 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cde51b-a14f-4713-86ed-a04c21e64892 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.187227 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1439/2881] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:21.455518 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.457482 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:21.469887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b4159-eb0e-449b-b561-b516cc405f64 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.474566 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.475252 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.475618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.475774 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] PATH_INFO: `/v3/projects/e6c1e44c9b4b439dabfc60cf9771b38f` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.477029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c1e44c9b4b439dabfc60cf9771b38f)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.478958 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.524257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c1e44c9b4b439dabfc60cf9771b38f', 'request_id': 'req-7a7e34a6-4fd5-44f2-bda5-756831c48b49'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:21.530701 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c1e44c9b4b439dabfc60cf9771b38f', 'request_id': 'req-7a7e34a6-4fd5-44f2-bda5-756831c48b49'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:21.538020 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c1e44c9b4b439dabfc60cf9771b38f', 'request_id': 'req-7a7e34a6-4fd5-44f2-bda5-756831c48b49'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:21.548954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] The token cache is being invalidate because project e6c1e44c9b4b439dabfc60cf9771b38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:15:21.549301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:21.550017 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-7a7e34a6-4fd5-44f2-bda5-756831c48b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:21.551569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1443/2882] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:15:21 2022] DELETE /identity/v3/projects/e6c1e44c9b4b439dabfc60cf9771b38f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:21.556397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.556684 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.556964 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.576482 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:21.577687 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:21.594747 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:21.637387 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1440/2883] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:21.644851 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.645133 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.645407 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.667609 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:21.668795 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:21.676840 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1444/2884] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:21.680941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.690345 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a4dff-c87b-4192-81a2-a5f601a04da3 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.694813 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.695400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.695673 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.695938 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.696569 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:15:21.696911 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.699161 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.710379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1441/2885] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:15:21.714907 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.728605 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f10e2-1a50-4464-881d-fe1e8d9cbe23 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.733412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.734030 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.734297 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.734565 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.735182 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.739607 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.768522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1445/2886] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:15:21 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:15:21.772699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.783389 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804cd79-6822-4918-9f0a-b1fad26145a7 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.838039 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.838696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.838984 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.839259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.839917 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.841824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.862941 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1442/2887] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:15:21 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:15:21.867019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.877627 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee59129-e2b5-44ca-abe3-e5bc3eec4984 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.882442 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.883081 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.883367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.883624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.884270 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.886520 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.896185 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1446/2888] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:21.899760 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.909648 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dba77f-a1f5-4b72-971e-1d6281528098 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.914200 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad3d99-cc13-4797-974a-29b7228662f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.914781 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad3d99-cc13-4797-974a-29b7228662f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.915058 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad3d99-cc13-4797-974a-29b7228662f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.915327 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad3d99-cc13-4797-974a-29b7228662f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.915956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad3d99-cc13-4797-974a-29b7228662f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.917556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad3d99-cc13-4797-974a-29b7228662f8 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.927874 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1443/2889] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:21.931692 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:21.941694 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd48b081-60be-4ce8-be32-8cc1d1785044 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:21.946198 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:21.946862 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.947172 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.947386 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] PATH_INFO: `/v3/projects/bf62731ccfeb4b9eab01f73d937c724d/users/6b86ec9003dd441ab1ee271679e12bc0/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:21.962074 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf62731ccfeb4b9eab01f73d937c724d, user_id=6b86ec9003dd441ab1ee271679e12bc0, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:21.964663 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:21.976658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1447/2890] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:15:21 2022] PUT /identity/v3/projects/bf62731ccfeb4b9eab01f73d937c724d/users/6b86ec9003dd441ab1ee271679e12bc0/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:21.986698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:21.986998 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:21.987272 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.019803 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] MFA Rules not processed for user `6b86ec9003dd441ab1ee271679e12bc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:22.021035 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:22.084061 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1444/2891] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:22.088569 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.108940 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:22.150384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da50ac-812f-4e2d-99af-29d35feab9f0 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.155180 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.155771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.156044 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.156310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.156924 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.158824 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.185161 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1448/2892] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:15:22 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:15:22.189261 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.199883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d41d14-3c94-4cd4-a9da-7e0c380b9cc6 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.211809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:22.256036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.256636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.256914 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.257184 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.257801 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.259736 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.280087 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1445/2893] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:15:22 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:15:22.284012 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.294499 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ebdc91-d2cf-4a65-bd2f-ee4979468645 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.299036 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.299620 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.299892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.300159 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.300781 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.302389 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.313056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1449/2894] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:22.316796 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.326725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79be76-9c40-4292-85c0-93822d0f7963 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.331293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd37c54-f83d-45a3-b4e6-932646bad921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.331907 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd37c54-f83d-45a3-b4e6-932646bad921 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.332185 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd37c54-f83d-45a3-b4e6-932646bad921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.332456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd37c54-f83d-45a3-b4e6-932646bad921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.333091 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd37c54-f83d-45a3-b4e6-932646bad921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.334732 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd37c54-f83d-45a3-b4e6-932646bad921 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.344361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1446/2895] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:22.348216 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.359155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f338ff4b-b916-433b-a8a9-f26ba6de0965 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.363718 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.364360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.364636 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.364906 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] PATH_INFO: `/v3/projects/484785b7a74e4baa9b50ba1079d252ad/users/81d9c2305bef4f77a59fae5d35fb701c/roles/6768099ea599447f9de57814f623441c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.379576 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=484785b7a74e4baa9b50ba1079d252ad, user_id=81d9c2305bef4f77a59fae5d35fb701c, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.382201 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.394742 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1450/2896] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:15:22 2022] PUT /identity/v3/projects/484785b7a74e4baa9b50ba1079d252ad/users/81d9c2305bef4f77a59fae5d35fb701c/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:15:22.406360 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.406752 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.407031 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.435630 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] MFA Rules not processed for user `81d9c2305bef4f77a59fae5d35fb701c`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:22.436821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:22.499527 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1447/2897] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 24 05:15:22.504041 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.563869 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a131fb-6f48-4269-a093-3ce406646101 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.568585 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.569179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.569453 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.569717 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.570362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.572234 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.598089 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1451/2898] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:15:22 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:15:22.602330 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.613315 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be3472-b733-40f5-93ae-05b513881835 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.670653 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.671274 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.671581 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.671855 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.672514 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.674503 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.695121 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1448/2899] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:15:22 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 24 05:15:22.699193 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.709227 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44f28b-061f-4ccb-ae39-13f8bfaa68ee admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.714367 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.714954 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.715230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.715496 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.716123 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.717733 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.729703 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1452/2900] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:22.733622 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.743646 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fca39c-49d5-4edb-88cf-f775315944c3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.748361 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.749008 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.749285 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.749608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] PATH_INFO: `/v3/projects/73f8ed6771ab4dfdbedc2d07463ce9f3/users/aeb7e7517ffc48818c4d04a7a0db30e3/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.761790 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:22.765162 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f8ed6771ab4dfdbedc2d07463ce9f3, user_id=aeb7e7517ffc48818c4d04a7a0db30e3, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.768458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.782386 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1449/2901] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:15:22 2022] PUT /identity/v3/projects/73f8ed6771ab4dfdbedc2d07463ce9f3/users/aeb7e7517ffc48818c4d04a7a0db30e3/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:15:22.786373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.796963 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df7ea2-6d0b-4ed9-9853-52ab0c39c373 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.836683 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:22.853070 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1678ac-abed-4cbe-90fe-466d2859b0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.853696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1678ac-abed-4cbe-90fe-466d2859b0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.853971 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1678ac-abed-4cbe-90fe-466d2859b0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.854848 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1678ac-abed-4cbe-90fe-466d2859b0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.855478 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1678ac-abed-4cbe-90fe-466d2859b0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.857059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1678ac-abed-4cbe-90fe-466d2859b0a0 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.866542 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1453/2902] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:22.870821 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:22.881023 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264222df-1a4c-4b49-a72c-6a22ba33a5e3 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:22.885595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:22.886276 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.886556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.886829 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] PATH_INFO: `/v3/projects/73f8ed6771ab4dfdbedc2d07463ce9f3/users/aeb7e7517ffc48818c4d04a7a0db30e3/roles/f3cd0a39225e41639953039b68ac4c88` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.888726 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f8ed6771ab4dfdbedc2d07463ce9f3, user_id=aeb7e7517ffc48818c4d04a7a0db30e3, role_id=f3cd0a39225e41639953039b68ac4c88)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:22.891440 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:22.901910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1450/2903] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:15:22 2022] PUT /identity/v3/projects/73f8ed6771ab4dfdbedc2d07463ce9f3/users/aeb7e7517ffc48818c4d04a7a0db30e3/roles/f3cd0a39225e41639953039b68ac4c88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:22.912101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:22.912390 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:22.912666 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:22.940102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] MFA Rules not processed for user `aeb7e7517ffc48818c4d04a7a0db30e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:22.941298 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:23.007955 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1454/2904] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:23.019638 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:23.021400 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:23.085382 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06bc15-5b10-4597-8e8c-6492d1b29899 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:23.089664 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:23.090295 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:23.090573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:23.090840 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:23.144036 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:23.145458 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:23.154955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1451/2905] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:24.660101 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:24.661549 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:24.674029 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12bb25-fce8-4dc1-bd8a-04f5ad11e147 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:24.677491 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:24.678104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:24.678380 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:24.678649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:24.732501 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:24.733850 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:24.742365 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1455/2906] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:25.253600 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:25.262738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e467a5-28d4-4d59-879b-6c62ffcdb4ee service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:25.266580 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede32df9-7e9f-4805-844f-e442d4fcf2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:25.267215 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede32df9-7e9f-4805-844f-e442d4fcf2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:25.267494 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede32df9-7e9f-4805-844f-e442d4fcf2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:25.267783 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede32df9-7e9f-4805-844f-e442d4fcf2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:25.278144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede32df9-7e9f-4805-844f-e442d4fcf2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:25.279824 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede32df9-7e9f-4805-844f-e442d4fcf2a8 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:25.289301 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1452/2907] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:29.152373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:29.212625 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b65802-cda8-4890-9ba6-1f31e03c40d0 service cinder] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:29.217852 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:29.218505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:29.218792 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:29.219054 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] PATH_INFO: `/v3/users/6b86ec9003dd441ab1ee271679e12bc0` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:29.220162 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b86ec9003dd441ab1ee271679e12bc0)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:29.222082 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:29.226877 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=100 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:29.271344 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b86ec9003dd441ab1ee271679e12bc0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.280274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b86ec9003dd441ab1ee271679e12bc0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.281052 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b86ec9003dd441ab1ee271679e12bc0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.281654 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b86ec9003dd441ab1ee271679e12bc0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.288441 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b86ec9003dd441ab1ee271679e12bc0'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.304993 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1456/2908] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:15:29 2022] DELETE /identity/v3/users/6b86ec9003dd441ab1ee271679e12bc0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:29.310233 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:29.310738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:29.310843 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:29.329969 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:29.331300 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:29.388930 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1453/2909] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:29.398556 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:29.399931 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:29.456512 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f851de-089c-4473-b45c-3152d3f09894 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:29.460155 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:29.460754 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:29.461040 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:29.461312 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:29.470149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:29.471571 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:29.481319 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1457/2910] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:29.739188 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:29.740711 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:29.753441 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b20061-d526-4c34-b45b-cb297ab27638 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:29.758054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:29.758699 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:29.758980 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:29.759255 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] PATH_INFO: `/v3/projects/bf62731ccfeb4b9eab01f73d937c724d` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:29.760536 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf62731ccfeb4b9eab01f73d937c724d)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:29.762696 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:29.779259 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:29.811493 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf62731ccfeb4b9eab01f73d937c724d', 'request_id': 'req-f9758e83-b12b-4fbb-831b-a43583664880'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.818785 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf62731ccfeb4b9eab01f73d937c724d', 'request_id': 'req-f9758e83-b12b-4fbb-831b-a43583664880'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.824350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf62731ccfeb4b9eab01f73d937c724d', 'request_id': 'req-f9758e83-b12b-4fbb-831b-a43583664880'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.835175 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] The token cache is being invalidate because project bf62731ccfeb4b9eab01f73d937c724d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:15:29.835468 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.836103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.837614 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1454/2911] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:15:29 2022] DELETE /identity/v3/projects/bf62731ccfeb4b9eab01f73d937c724d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:15:29.841736 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:29.851288 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca89af-a436-4a73-acbd-f41bf15b4b21 service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:29.906536 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:29.907084 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:29.907361 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:29.907633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] PATH_INFO: `/v3/users/81d9c2305bef4f77a59fae5d35fb701c` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:29.908732 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d9c2305bef4f77a59fae5d35fb701c)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:29.910675 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:29.939104 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:29.956903 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d9c2305bef4f77a59fae5d35fb701c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.963871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d9c2305bef4f77a59fae5d35fb701c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.964480 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d9c2305bef4f77a59fae5d35fb701c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.965080 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d9c2305bef4f77a59fae5d35fb701c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.972164 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d9c2305bef4f77a59fae5d35fb701c'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:29.989601 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1458/2912] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:15:29 2022] DELETE /identity/v3/users/81d9c2305bef4f77a59fae5d35fb701c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:30.251961 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:30.253446 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:30.266815 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9758e83-b12b-4fbb-831b-a43583664880 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:30.323226 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:30.323908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:30.324173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:30.324449 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] PATH_INFO: `/v3/projects/484785b7a74e4baa9b50ba1079d252ad` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:30.325595 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=484785b7a74e4baa9b50ba1079d252ad)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:30.327551 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:30.372933 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '484785b7a74e4baa9b50ba1079d252ad', 'request_id': 'req-f4c106ae-4646-4c05-9a1a-46d732dc93cd'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.379593 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '484785b7a74e4baa9b50ba1079d252ad', 'request_id': 'req-f4c106ae-4646-4c05-9a1a-46d732dc93cd'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.385464 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '484785b7a74e4baa9b50ba1079d252ad', 'request_id': 'req-f4c106ae-4646-4c05-9a1a-46d732dc93cd'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.396903 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] The token cache is being invalidate because project 484785b7a74e4baa9b50ba1079d252ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:15:30.397187 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.397844 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.399340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1455/2913] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:15:30 2022] DELETE /identity/v3/projects/484785b7a74e4baa9b50ba1079d252ad => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:15:30.403895 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:30.405532 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:30.418130 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad5148-6b75-49b5-a357-358349953305 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:30.472988 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:30.473615 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:30.473890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:30.474213 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] PATH_INFO: `/v3/users/aeb7e7517ffc48818c4d04a7a0db30e3` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:30.475699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb7e7517ffc48818c4d04a7a0db30e3)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:30.477699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:30.522323 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb7e7517ffc48818c4d04a7a0db30e3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.531059 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb7e7517ffc48818c4d04a7a0db30e3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.531727 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb7e7517ffc48818c4d04a7a0db30e3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.532350 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb7e7517ffc48818c4d04a7a0db30e3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.540184 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-28336de3-2448-4c6f-b645-dc262f6892fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb7e7517ffc48818c4d04a7a0db30e3'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.557119 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1459/2914] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:15:30 2022] DELETE /identity/v3/users/aeb7e7517ffc48818c4d04a7a0db30e3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:30.839722 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:30.841292 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:30.854302 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c106ae-4646-4c05-9a1a-46d732dc93cd admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:30.887564 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:30.915712 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:30.916350 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:30.916631 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:30.916960 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] PATH_INFO: `/v3/projects/73f8ed6771ab4dfdbedc2d07463ce9f3` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:30.918099 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f8ed6771ab4dfdbedc2d07463ce9f3)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:30.920051 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:30.965103 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f8ed6771ab4dfdbedc2d07463ce9f3', 'request_id': 'req-9dabbb33-b401-4a70-a12f-da08169bd76b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.972239 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f8ed6771ab4dfdbedc2d07463ce9f3', 'request_id': 'req-9dabbb33-b401-4a70-a12f-da08169bd76b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.977800 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f8ed6771ab4dfdbedc2d07463ce9f3', 'request_id': 'req-9dabbb33-b401-4a70-a12f-da08169bd76b'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.988766 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] The token cache is being invalidate because project 73f8ed6771ab4dfdbedc2d07463ce9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:15:30.989054 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.989686 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-9dabbb33-b401-4a70-a12f-da08169bd76b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:15:30.991199 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1456/2915] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:15:30 2022] DELETE /identity/v3/projects/73f8ed6771ab4dfdbedc2d07463ce9f3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:15:30.996594 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:30.996899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:30.997168 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.016709 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:31.017846 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:31.036373 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:31.077734 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1460/2916] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:31.086678 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.086995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.087269 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.107805 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:31.109016 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:31.116949 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1457/2917] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:31.121019 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.130630 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169349e-3f22-4838-b41e-f29db1d73641 None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.135127 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.136211 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.136487 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.136757 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] PATH_INFO: `/v3/domains` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.137400 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 24 05:15:31.137696 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.139363 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.150658 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1461/2918] 10.222.0.50 () {58 vars in 1153 bytes} [Sat Sep 24 05:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 24 05:15:31.155738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.165677 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a87d2b7-2a15-4c2b-a986-f685559c139d None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.170707 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.171355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.171636 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.171910 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.172563 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.174526 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.202077 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1458/2919] 10.222.0.50 () {60 vars in 1153 bytes} [Sat Sep 24 05:15:31 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:15:31.206245 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.216586 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c31a25-a513-4ade-aa58-d3347fbaa137 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.271671 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.272322 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.272791 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.273068 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] PATH_INFO: `/v3/users` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.273688 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.275624 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.296771 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1462/2920] 10.222.0.50 () {60 vars in 1144 bytes} [Sat Sep 24 05:15:31 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 24 05:15:31.300819 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.311114 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac44e3-23dd-482a-bdb7-56244fc278bc admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.315731 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.316329 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.316607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.316879 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.317515 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.319155 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.329379 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1459/2921] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:31.333149 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.343645 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb5682-eba6-4f14-89c6-988bf2af0106 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.348308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d70f98-5ee2-4076-b0b5-88475b7786c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.348900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d70f98-5ee2-4076-b0b5-88475b7786c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.349179 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d70f98-5ee2-4076-b0b5-88475b7786c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.349452 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d70f98-5ee2-4076-b0b5-88475b7786c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.350230 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d70f98-5ee2-4076-b0b5-88475b7786c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.351827 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d70f98-5ee2-4076-b0b5-88475b7786c7 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.362274 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1463/2922] 10.222.0.50 () {58 vars in 1122 bytes} [Sat Sep 24 05:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 24 05:15:31.366294 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.376573 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474555da-f1ca-42a7-9566-1d8a6ed7fbb4 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.381219 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.381872 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.382194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.382474 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] PATH_INFO: `/v3/projects/0ce523f87745437e9c3d6630d1fb2e51/users/67e8948aca7b4a1da87d81830cc819fe/roles/6768099ea599447f9de57814f623441c` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.398456 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce523f87745437e9c3d6630d1fb2e51, user_id=67e8948aca7b4a1da87d81830cc819fe, role_id=6768099ea599447f9de57814f623441c)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.401106 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.416234 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1460/2923] 10.222.0.50 () {60 vars in 1483 bytes} [Sat Sep 24 05:15:31 2022] PUT /identity/v3/projects/0ce523f87745437e9c3d6630d1fb2e51/users/67e8948aca7b4a1da87d81830cc819fe/roles/6768099ea599447f9de57814f623441c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:15:31.421446 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b374b-989b-48cf-aa6d-e3275077971f None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.421950 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b374b-989b-48cf-aa6d-e3275077971f None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.422085 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772b374b-989b-48cf-aa6d-e3275077971f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.440475 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-772b374b-989b-48cf-aa6d-e3275077971f None None] MFA Rules not processed for user `eb0ba9d6a6bf45098fbc94fef46dc27d`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:31.441677 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-772b374b-989b-48cf-aa6d-e3275077971f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:31.500381 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1464/2924] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:31.510072 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:31.512238 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:31.567771 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cef21-3e49-493d-967f-906314636a8b admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:31.571340 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:31.571935 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:31.572210 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:31.572480 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:31.581779 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:31.583218 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:31.592370 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1461/2925] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:32.193505 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b78b63-43a5-4c54-b6bf-5c87760bb279 None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:32.193769 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b78b63-43a5-4c54-b6bf-5c87760bb279 None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:32.194056 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b78b63-43a5-4c54-b6bf-5c87760bb279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:32.223412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-16b78b63-43a5-4c54-b6bf-5c87760bb279 None None] MFA Rules not processed for user `67e8948aca7b4a1da87d81830cc819fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:32.224634 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-16b78b63-43a5-4c54-b6bf-5c87760bb279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:32.243806 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:32.295421 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1465/2926] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:32.305908 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:32.307325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:32.312556 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:32.367427 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5abdb-5c02-4483-b99c-9293e86879ea service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:32.371100 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a5e44f909a94c74990b2e80f1da5169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:32.371705 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:32.371981 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:32.372251 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:32.380530 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:32.381956 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:32.390637 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1462/2927] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:33.750649 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:33.751166 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:33.751290 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:33.769568 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] MFA Rules not processed for user `67e8948aca7b4a1da87d81830cc819fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:33.770767 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:33.778069 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1466/2928] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:34.013258 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:34.022692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679470a7-8adf-4908-807f-83b8e20ecdc1 service cinder] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:34.026810 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:34.027396 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:34.027668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:34.027932 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:34.029876 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:34.039154 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:34.040553 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:34.049209 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1463/2929] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:42.604257 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:42.613945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34864f38-22d1-4231-8e0a-0890dc6aaede None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:42.617557 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:42.618191 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:42.618471 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:42.618745 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:42.620502 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:42.691276 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:42.692705 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50f95-bbd1-47a8-a026-ce46c10a69d6 service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:42.703972 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1467/2930] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:44.045397 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=31 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:15:45.051418 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:15:45.059968 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe1d8c-5933-42b6-8561-28583cb49751 service neutron] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:15:45.063585 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:15:45.064169 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:45.064442 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:45.064710 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:45.066438 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:45.079293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:15:45.080700 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28735b85-dbc2-405d-abdd-a805f85559ed service neutron] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:15:45.091881 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1464/2931] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:15:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:15:50.380264 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] REQUEST_METHOD: `POST` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:15:50.380752 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:15:50.380866 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:15:50.399278 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.auth.core [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] MFA Rules not processed for user `67e8948aca7b4a1da87d81830cc819fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:15:50.400450 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:15:50.407900 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1468/2932] 10.222.0.50 () {58 vars in 958 bytes} [Sat Sep 24 05:15:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:15:55.086608 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=6 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:16:00.393856 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=58 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:16:12.127953 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62d5b0-85d9-41d3-a071-cb4e6c7f8dab None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:12.128491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62d5b0-85d9-41d3-a071-cb4e6c7f8dab None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:12.128491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62d5b0-85d9-41d3-a071-cb4e6c7f8dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:12.147668 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-6e62d5b0-85d9-41d3-a071-cb4e6c7f8dab None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:16:12.148864 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-6e62d5b0-85d9-41d3-a071-cb4e6c7f8dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:12.203561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1465/2933] 10.222.0.50 () {58 vars in 1009 bytes} [Sat Sep 24 05:16:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:16:12.248199 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:12.249653 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:12.318892 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfabee-06bb-4adb-b535-c325b332681e None None] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:12.323758 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:12.324362 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:12.324639 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:12.324909 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:12.333310 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:12.334914 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:12.343465 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1469/2934] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:16:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:16:22.197141 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=25 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:16:22.339893 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=32 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:16:27.166959 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] REQUEST_METHOD: `POST` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:27.167428 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:27.167539 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:27.186854 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.auth.core [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] MFA Rules not processed for user `8b1bc0d6bb8248cd86c8ac462709e326`. Rule list: `[]` (Enabled: `True`). {{(pid=81394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 24 05:16:27.188044 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:27.195393 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1466/2935] 10.222.0.50 () {58 vars in 967 bytes} [Sat Sep 24 05:16:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 24 05:16:27.208464 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:27.219618 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d65c976-232f-4e1b-bb20-eacd5c2aea32 service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:27.224686 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b8a56d415755492fae33fc13f57c8f3c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:27.225283 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:27.225563 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:27.225837 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:27.234412 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:27.235977 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:27.243890 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1470/2936] 10.222.0.50 () {60 vars in 1384 bytes} [Sat Sep 24 05:16:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 24 05:16:29.071365 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:29.072799 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:29.129394 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:29.142330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92de982-4caa-45a9-908b-cc71c53fe654 service swift] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:29.147235 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:29.147570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c808ed-0069-4b19-8172-f651b7363679 None None] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:29.147855 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:29.148784 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:29.149064 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] PATH_INFO: `/v3/users/67e8948aca7b4a1da87d81830cc819fe` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:29.150204 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67e8948aca7b4a1da87d81830cc819fe)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:29.152099 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:29.152697 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:29.153308 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:29.153586 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:29.154681 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:29.156629 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:29.168506 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:29.170725 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:29.181650 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1467/2937] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:16:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:16:29.201150 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67e8948aca7b4a1da87d81830cc819fe'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.208796 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67e8948aca7b4a1da87d81830cc819fe'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.209527 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67e8948aca7b4a1da87d81830cc819fe'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.210209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67e8948aca7b4a1da87d81830cc819fe'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.217952 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67e8948aca7b4a1da87d81830cc819fe'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.235887 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1471/2938] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:16:29 2022] DELETE /identity/v3/users/67e8948aca7b4a1da87d81830cc819fe => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:16:29.408858 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:29.419339 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55af1-6ba6-4521-b13f-a33b39097ec6 service placement] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:29.475995 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:29.476623 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:29.476901 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:29.477173 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] PATH_INFO: `/v3/projects/0ce523f87745437e9c3d6630d1fb2e51` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:29.478325 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce523f87745437e9c3d6630d1fb2e51)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:29.480491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:29.525692 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce523f87745437e9c3d6630d1fb2e51', 'request_id': 'req-1373d4d1-064a-493e-86ff-4f66e5e152a4'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.533624 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce523f87745437e9c3d6630d1fb2e51', 'request_id': 'req-1373d4d1-064a-493e-86ff-4f66e5e152a4'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.539150 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce523f87745437e9c3d6630d1fb2e51', 'request_id': 'req-1373d4d1-064a-493e-86ff-4f66e5e152a4'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.549873 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] The token cache is being invalidate because project 0ce523f87745437e9c3d6630d1fb2e51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:16:29.550194 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.550825 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:16:29.552280 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1468/2939] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:16:29 2022] DELETE /identity/v3/projects/0ce523f87745437e9c3d6630d1fb2e51 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:16:39.231642 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:16:39.547032 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:16:53.940240 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:53.941661 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:54.021826 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407f955-7c99-49bf-ad79-144cd4d55b85 admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:54.078725 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:54.079360 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:54.079633 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:54.079899 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:54.081665 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:54.091548 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:54.092976 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:54.101256 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1472/2940] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:16:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:16:56.599818 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:16:56.603561 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:56.689455 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373d4d1-064a-493e-86ff-4f66e5e152a4 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:16:56.694866 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2451d53fb9974222827b6520b8bce5f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:16:56.695463 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:16:56.695738 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:16:56.696010 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:16:56.697698 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:16:56.709955 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:16:56.711543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:16:56.722601 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1469/2941] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:16:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:17:04.097436 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=46 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:17:05.319330 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:17:05.320871 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:17:05.331936 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b3966-a46a-40ac-a01f-46836b56d76f service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:17:05.335448 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '266e3f374e794afca24caf914399a7d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:17:05.336035 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] REQUEST_METHOD: `GET` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:17:05.336308 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:17:05.336575 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:17:05.338259 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:17:05.399065 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:17:05.400472 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:17:05.410889 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1473/2942] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:17:06.718170 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=34 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:17:11.646848 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:17:11.648528 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:17:11.663607 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44c06b-de38-431b-8cab-758c0905ec37 service nova] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:17:11.669168 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:17:11.669812 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:17:11.670144 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:17:11.670590 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] PATH_INFO: `/v3/users/60edcebcccaa4da99ffddadd3e27c1c0` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:17:11.671789 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60edcebcccaa4da99ffddadd3e27c1c0)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:17:11.673721 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:17:11.721316 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60edcebcccaa4da99ffddadd3e27c1c0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:11.728724 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60edcebcccaa4da99ffddadd3e27c1c0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:11.729424 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60edcebcccaa4da99ffddadd3e27c1c0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:11.737196 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60edcebcccaa4da99ffddadd3e27c1c0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:11.737828 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60edcebcccaa4da99ffddadd3e27c1c0'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:11.755570 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1470/2943] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:17:11 2022] DELETE /identity/v3/users/60edcebcccaa4da99ffddadd3e27c1c0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:17:11.934384 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:17:11.945301 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52883698-5649-4eb5-8770-62a0753970ab service neutron] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:17:12.012128 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:17:12.012756 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:17:12.013034 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:17:12.013311 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] PATH_INFO: `/v3/projects/fef19544ec414094953daea6d545a361` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:17:12.015066 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef19544ec414094953daea6d545a361)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:17:12.016945 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:17:12.062408 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef19544ec414094953daea6d545a361', 'request_id': 'req-ea0f9e85-9093-43c4-bb00-82541b745a1a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.067888 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef19544ec414094953daea6d545a361', 'request_id': 'req-ea0f9e85-9093-43c4-bb00-82541b745a1a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.075022 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef19544ec414094953daea6d545a361', 'request_id': 'req-ea0f9e85-9093-43c4-bb00-82541b745a1a'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.086398 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] The token cache is being invalidate because project fef19544ec414094953daea6d545a361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:17:12.086619 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.087242 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.088711 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1474/2944] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:17:11 2022] DELETE /identity/v3/projects/fef19544ec414094953daea6d545a361 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:17:12.092787 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:17:12.094222 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:17:12.107247 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875518-c3aa-4bf3-8acd-29e3082fa123 admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:17:12.161491 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:17:12.162156 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:17:12.162435 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:17:12.162703 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] PATH_INFO: `/v3/users/e0b2e05dd2694583b70c8914519ac343` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:17:12.163809 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b2e05dd2694583b70c8914519ac343)` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:17:12.165719 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:17:12.211740 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b2e05dd2694583b70c8914519ac343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.218298 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b2e05dd2694583b70c8914519ac343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.218922 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b2e05dd2694583b70c8914519ac343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.225891 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b2e05dd2694583b70c8914519ac343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.226549 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.notifications [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b2e05dd2694583b70c8914519ac343'} {{(pid=81394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.234748 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:17:12.245293 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1471/2945] 10.222.0.50 () {58 vars in 1224 bytes} [Sat Sep 24 05:17:12 2022] DELETE /identity/v3/users/e0b2e05dd2694583b70c8914519ac343 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 24 05:17:12.412037 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Authenticating user token {{(pid=81393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:17:12.413531 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.fernet_utils [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:17:12.426930 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f9e85-9093-43c4-bb00-82541b745a1a admin admin] Validating token access rules against request {{(pid=81393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:17:12.447292 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:17:12.484934 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb0ba9d6a6bf45098fbc94fef46dc27d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9c9e5a50c664c33aa419611e20c9c00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:17:12.485522 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:17:12.485799 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:17:12.486102 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] PATH_INFO: `/v3/projects/edba865cd0f14c16ba0060b3c031cefa` {{(pid=81393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:17:12.487207 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edba865cd0f14c16ba0060b3c031cefa)` {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:17:12.489209 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] RBAC: Authorization granted {{(pid=81393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:17:12.533699 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edba865cd0f14c16ba0060b3c031cefa', 'request_id': 'req-27384d48-097d-44db-8075-13693fc5f4b2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.539120 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edba865cd0f14c16ba0060b3c031cefa', 'request_id': 'req-27384d48-097d-44db-8075-13693fc5f4b2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.546524 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edba865cd0f14c16ba0060b3c031cefa', 'request_id': 'req-27384d48-097d-44db-8075-13693fc5f4b2'} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.557267 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] The token cache is being invalidate because project edba865cd0f14c16ba0060b3c031cefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 24 05:17:12.557577 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.558228 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG keystone.notifications [None req-27384d48-097d-44db-8075-13693fc5f4b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 24 05:17:12.559721 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: [pid: 81393|app: 0|req: 1475/2946] 10.222.0.50 () {58 vars in 1233 bytes} [Sat Sep 24 05:17:12 2022] DELETE /identity/v3/projects/edba865cd0f14c16ba0060b3c031cefa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 24 05:17:22.243621 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:17:22.554243 dv-cinder-859186-1-iscsi devstack@keystone.service[81393]: DEBUG dbcounter [-] [81393] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 24 05:18:57.754882 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Authenticating user token {{(pid=81394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 24 05:18:57.756299 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:18:57.890899 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452220f-a377-441c-890f-5f978066bbba admin admin] Validating token access rules against request {{(pid=81394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 24 05:18:57.953014 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdf673c05e4443e8bce9b9d7371d1fd7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '308491fe36314e479f19ead78bb4b5a6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 24 05:18:57.953543 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] REQUEST_METHOD: `GET` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 24 05:18:57.953823 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] SCRIPT_NAME: `/identity` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 24 05:18:57.954127 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 24 05:18:57.955839 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.fernet_utils [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 24 05:18:57.967317 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 24 05:18:57.968883 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d81aa-df17-467f-bef0-7d53061455fb service placement] RBAC: Authorization granted {{(pid=81394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 24 05:18:57.979355 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: [pid: 81394|app: 0|req: 1472/2947] 10.222.0.50 () {60 vars in 1365 bytes} [Sat Sep 24 05:18:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 24 05:19:07.975098 dv-cinder-859186-1-iscsi devstack@keystone.service[81394]: DEBUG dbcounter [-] [81394] Writing DB stats keystone:SELECT=89 {{(pid=81394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}